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Overview 



An investment in postsecondary education has been repeatedly shown to pay high monetary and 
nonmonetary dividends to students and society at large. Despite such benefits, research shows that 
close to half of all students who matriculate at a community college drop out before graduating 
and do not complete a degree at any other college or university within a six-year time frame (U.S. 
Department of Education, 2003b). The reasons for this are many, ranging from weak academic 
preparation to difficulties balancing work, family, and school obligations. 

To study the effect of supplemental financial aid with an incentive component to encourage 
academic success and persistence, two New Orleans-area colleges operated a performance-based 
scholarship program with counseling in 2004-2005. The program was targeted to low-income 
parents as part of MDRC’s multisite Opening Doors demonstration. With funding from the 
Louisiana Department of Social Services and the Louisiana Workforce Commission, the colleges 
offered students $1,000 for each of two semesters ($2,000 total) — distributed in three separate 
payments each semester — if they met two conditions: They had to enroll in college at least half 
time and they had to maintain an average grade of “C” or better. Students did not have to be 
welfare recipients, and the scholarships were paid in addition to federal Pell Grants. Program 
counselors monitored whether students met benchmarks, and physically handed the students their 
checks at the beginning, middle, and end of the semester. 

Using a random assignment design — the “gold standard” methodology in program evaluation — 
MDRC assigned 1,019 parents who were enrolled or planning to enroll in a community college to 
either a control group who received their college’s standard financial aid package and student 
services or to a program group who received the same standard aid package and student services in 
addition to being eligible for the Opening Doors performance-based scholarship. Analyses in this 
report show that: 

• The Opening Doors program encouraged more students to register for college. Students 
who received the scholarship were not only more likely (by 5.3 percentage points) to register, 
they were more likely (by 6.4 percentage points) to register full time, although only half-time 
enrollment was required to maintain the scholarship. 

• The program increased persistence. Longer-term analyses for the first groups of students 
who entered the Opening Doors study show that program group students were more likely (by 
6.5 percentage points) to be registered through four semesters after random assignment. 

• The program increased the number of credits that students earned. Lollow-up data on 
the first groups of students to enter the Opening Doors study show positive effects on credit 
accumulation and grades through the fourth semester after random assignment. 

• The program had positive impacts on a range of social and psychological outcomes. 

Students in the Opening Doors program reported greater engagement in working toward their 
personal goals and higher levels of perceived social support. 

Tragically, Hurricane Katrina, a category five hurricane, hit the Gulf Coast in August 2005, 
intermpting the follow-up period of the study. However, MDRC has made it a priority to replicate 
the program (and variations of it) in order to build more evidence on the potential of performance- 
based scholarships to help at-risk students through its recently launched Performance-Based 
Scholarship demonstration. 
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Preface 



College degrees open the door to economic and social opportunity. Since the passage of 
the Higher Education Act of 1965, the federal government has invested heavily in programs 
designed to make college affordable to all Americans. A growing body of research demon- 
strates that college-educated adults earn more, lead healthier lives, and are more engaged in civ- 
ic life than adults who have only a high school education or less. Community colleges, in par- 
ticular, provide opportunities to low-income individuals, including people of color, immigrants, 
full- and part-time workers, and students who are the first in their families to attend college. 

While college enrollments in the United States have risen steadily over the years, rates 
of college completion have remained flat. About two-thirds of students who begin at a four-year 
college or university earn a bachelor’s degree within five years. For students who begin at 
community colleges, about one-third earn an associate’s degree or a bachelor’s degree within 
six years. Community college students, in particular, are stymied by multiple challenges, among 
them weak academic preparation; difficulties balancing work, family, and school obligations; 
and lack of adequate financial aid. 

This report describes positive short-term findings from a study of a program at two 
Louisiana community colleges that sought to confront some of these problems head on through 
performance-based scholarships combined with counseling. The low-income, mostly single 
mothers in the program were more likely to register full time, to persist in their studies from 
semester to semester, to earn higher grades and more credits, and to have better social and psy- 
chological outcomes than students in a control group. Unfortunately, Hurricane Katrina tempo- 
rarily shut down the two colleges during the study, constraining MDRC’s ability to measure the 
program’s long-term effects. 

Nevertheless, these promising results in Louisiana have spurred interest among other 
states and institutions to test variations of performance-based scholarships. Recognizing that 
Louisiana is just one study — and that the results may not be easily generalized to other popula- 
tions — MDRC recently launched a national Performance-Based Scholarship demonstration, 
with support from the Bill & Melinda Gates Foundation and other funders. Colleges and univer- 
sities in California, New York, Ohio, and New Mexico are participating. We look forward to 
learning much more about this promising form of scholarship, which offers an appealing way to 
increase financial aid to the neediest students. 



Gordon Berlin 
President 
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Executive Summary 



An investment in postsecondary education has been repeatedly shown to pay high mon- 
etary and nonmonetary dividends to students and society at large. In 2006, college graduates 
were about one-half as likely to be unemployed compared with high school dropouts, and about 
one-fourth as likely to live in poverty compared with high school graduates.' A college educa- 
tion may confer nonmonetary benefits as well. For example, studies have found that adults who 
have postsecondary education are more likely to be in good health, participate in other civic af- 
fairs, and have children who do well in school than adults who have not attended college.^ 

The nation’s 1,200 community colleges — with their open-admissions policies and 
relatively low cost — provide a pathway to educational and economic opportunity for millions 
of adults. Yet, despite the benefits of completing a postsecondary education, research shows that 
of all students who matriculate at community college, about one-third actually earn an asso- 
ciate’s degree or a bachelor’s degree within six years.^ The reasons for this are many, including 
weak academic preparation; difficulties balancing work, family, and school obligations; and 
lack of adequate financial aid. 

MDRC launched the Opening Doors demonstration to test four distinct interventions 
that were designed to help more students persist in community college and accomplish their 
academic and personal goals. In Louisiana — the subject of this report — researchers examined 
the effects of a performance-based scholarship with counseling targeted to low-income parents 
who were enrolled in or planning to enroll in community college. The performance-based schol- 
arship offered $1,000 for each of two semesters ($2,000 total) if students met two conditions: 
They had to enroll in college at least half time and they had to maintain an average grade of “C” 
or better, equivalent to a grade point average (GPA) of 2.0. The scholarships were paid in addi- 
tion to Pell Grants and any other financial aid for which students qualified, and were paid in 
increments, three times each semester, so that program counselors could monitor students’ 
compliance with these benchmarks and physically hand the students their checks at the begin- 
ning, middle, and end of the semester. Funding for the scholarships was provided by the Louisi- 
ana Department of Social Services and the Louisiana Workforce Commission, and consisted of 



'The College Board, Trends in College Pricing (Washington, DC, 2007). 

^Thomas S. Dee, “Are There Civic Returns to Education?,” Journal of Public Economics 88, 9 (August 
2004): 1697-1720; Gina Kolata, “The New Age: A Surprising Secret to a Long Life: Stay in School,” The New 
York Times (January 3, 2007). 

^U.S. Department of Education, National Center for Education Statistics, Community College Students: 
Goals, Academic Preparation, and Outcomes (Washington, DC, 2003). 
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flexible funds from the Temporary Assistance for Needy Families (TANF) program/ Despite 
the funding source, students did not have to be welfare recipients. 

MDRC and its partners in The Network on Transitions to Adulthood, funded by the 
MacArthur Foundation, examined the effects of the performance-based scholarship on stu- 
dents’ registration in college, course taking and completion, persistence, and other outcomes. 
Students were randomly assigned either to a control group that received standard financial aid 
and other college services or to a program group that received the same benefits in addition to 
being eligible for the performance-based scholarship with counseling. By comparing academic 
progress and other outcomes over time, researchers are able to determine what difference, or 
impact, the scholarship made in students’ lives. The evaluation took place at two institutions 
located in the New Orleans area: Delgado Community College and the Louisiana Technical 
College-West Jefferson. 

The Opening Doors program in Louisiana began in spring 2004 and operated through 
summer 2005. Just after the program ended. Hurricane Katrina sfruck the Gulf Coast region and 
caused severe and widespread destmction. The two colleges involved in the study shut down 
temporarily, and many students in the study moved away. To produce a fair assessment of the 
scholarship’s effects, this report focuses mainly on outcomes before the storm, although forth- 
coming research by members of The Network on Transitions to Adulthood will say more about 
how students fared after the hurricane. 

Key findings from this report include the following: 

• The Opening Doors program encouraged more students to register for 
college. Although the program was marketed mainly to students who had al- 
ready taken steps to enroll in college, it resulted in higher registration (by 5.3 
percentage points, or 7 percent) among program group students in the first 
semester. In addition, students eligible for the scholarship were more likely, 
by 6.4 percentage points, to register full time, although only half-time 
enrollment was required to maintain the scholarship. 

• The program increased persistence. The second semester after random as- 
signment, students in the program group were more likely (by 15 percentage 
points, or 30 percent) to register than students in the control group. Longer- 
term analyses for the first groups of students to enter the Opening Doors 
study (and who were able to make more progress in school before Hurricane 



"^Like many other states, Louisiana amassed a large funding surplus in its TANF program in the wake of 
rapid declines in welfare caseloads. The federal government permitted states to use TANF funds in many ways, 
as long as they were used to help low-income families achieve self-sufficiency. 
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Katrina) show that program group students were more likely to enroll in col- 
lege, by 6.5 percentage points, up to four semesters after random assignment, 
although the results diminish somewhat over time. 

• The program increased the number of credits that students earned. Stu- 
dents in the program group attempted and earned more credits, both in college- 
level courses and in developmental-level courses (for example, basic English 
and math), and were more likely to meet the GPA benchmark set by the pro- 
gram. Follow-up data on the first groups of students to enter the Opening 
Doors study show that program group students earned 3.5 more credits than 
control group students through the fourth semester after random assignment. 

• The program had positive impacts on a range of social and psychologi- 
cal outcomes. Sample members were surveyed about 16 months after they 
entered the study, on average. Students who received scholarships reported 
more positive feelings about themselves and their ability to accomplish their 
long-term goals. They also indicated that they had more social support and 
were more politically engaged than students who did not get scholarships. 



Who Was Served by the Program? 

As noted above, Louisiana’s Opening Doors program was funded through surplus state- 
level TANF funds. The funding source made it necessary to designate the scholarship for a 
population that is similar to the one served by TANF — namely, parents of at least one depen- 
dent child under 19 who had a household income under 200 percent of the federal poverty level. 
(Receipt of welfare was not a requirement.) To be eligible for the study, students also had to be 
between the ages of 18 and 34; have a high school diploma. General Educational Development 
(GED) certificate, or a passing score on a college entrance examination; and not already have a 
degree or certificate from an accredited college or university. 

Program staff reemited low-income parents for the study mainly during college orienta- 
tion and testing sessions, although they also put up flyers around campus and advertised in the 
community. Over the course of four semesters, a little more than 1,000 students agreed to par- 
ticipate in the study and were randomly assigned to program and control groups in four separate 
cohorts based on the semester in which they registered: spring 2004, summer 2004, fall 2004, 
and spring 2005. Over 90 percent of the sample members were women, and more than four out 
of five were black. The average age of students in the sample was 25, and the average age of 
their youngest child was just over 3 years old. Only 10 percent of the sample received TANF 
assistance, but a majority received some type of government assistance — most often, food 
stamps. About half the students were working at the time they entered the study. 
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How Was the Program Implemented? 

The performance-based scholarship was paid to students on the condition that they met 
certain requirements. Students received $250 upon enrolling at least half time (defined as six or 
more credit hours per semester); $250 after midterms, contingent on staying enrolled at least 
half time and earning passing grades (that is, a C average); and $500 upon completion of 
courses with a C average or better. This offer was extended for two semesters, giving students 
the opportunity to receive up to $2,000 in total. The program tried to create a strong incentive 
for students to do well and to receive the full amount of money that was offered. Hence, if stu- 
dents did not do well at midterms but met the grade point average at the end of the semester, 
they could still recoup the full amount of the scholarship. Similarly, students did not have to 
enroll for two consecutive terms to qualify for the full amount, but could take off a semester and 
return later if they chose (though this option was no longer available after summer 2005, when 
the program shut down). 

As the program was first conceived, MDRC hoped that the counselors would get to 
know the students on a personal level, encourage them to stay in school and make good 
progress, and take an active role in making referrals to appropriate services on campus or in the 
community if academic or personal problems interfered with school. Some counselors came 
closer to fulfilling this vision than others, but as a group, the counselors played mostly a moni- 
toring function. Counselors met with Opening Doors students periodically to monitor their aca- 
demic progress and disburse checks. They generally kept their meetings brief, and did not try to 
influence students’ choice of courses or delve into personal problems. The counselors used 
campus computer systems and contacted faculty to find out students’ enrollment status and 
grades, and coordinated with the bursar’s office to issue students’ checks, which they handed to 
students in person. Overall: 

• The program’s key features were fully implemented. 

Delgado Community College and Louisiana Technical College-West Jefferson had suf- 
ficient staff and procedures in place to ensure that the primary features of the Opening Doors 
program were generally executed as planned. While the counseling was less intensive than 
MDRC originally intended, it was adequate to deliver the scholarship. All students in the pro- 
gram group had the opportunity to receive the scholarship for two semesters. 

• There was a sharp contrast between the Opening Doors scholarship and 
other services available to students in the control group. 

In order to receive a “fair test,’’ a program must not only be fully implemented, but also 
differ from the services available to the control group. At the Louisiana colleges, both program 
group and control group members had access to regular financial aid (including federal Pell 
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Grants), but the control group did not have access to the Opening Doors scholarship or anything 
like it. In addition, although the counseling role in Opening Doors consisted mostly of brief 
meetings and monitoring of students’ enrollment and grades, students in the program group re- 
ported that they got more attention and support from college staff than control group students. 

• Nine out of ten students in the program group received at least one in- 
stallment of an Opening Doors scholarship. 

Over two semesters, the average amount of Opening Doors scholarship funds that stu- 
dents received was $1,133. Less than one-third of students qualified for the full $2,000 over two 
semesters. Program staff believed that the main reasons why more students did not receive the 
full scholarship included weak academic preparation for college, difficulty balancing school 
with work and parenting responsibilities, and health problems. 

• In qualitative interviews, several students in the program group spoke 
positively about their experience in Opening Doors. Both program 
group and control group members indicated strong motivation to earn a 
college degree, but also described significant barriers. 

Researchers affiliated with The Network on Transitions to Adulthood conducted 
intensive, individual interviews with 50 women from the Opening Doors sample before and 
after Hurricane Katrina stmek the area. Several women in the program group indicated that the 
scholarship helped them financially and that they appreciated the encouragement they got from 
program staff Another theme that emerged from the interviews was the challenge of juggling 
work, child care, and financial responsibilities while going to school. A number of women in 
the qualitative sample reported significant health problems, either for themselves or for family 
members in their care. 



How Was the Impact of the Program Evaluated? 

MDRC assigned students, prior to enrollment and at random, to either a program group 
or a control group to measure the impact of the Opening Doors program. Random assignment 
ensured that the academic backgrounds, motivation levels, and personal characteristics of the 
students in the two groups were similar at the start of the study. By tracking both groups of stu- 
dents over time and comparing their outcomes, researchers can determine what difference the 
program makes in registration, persistence, course credits earned, and other outcomes. Random 
assignment occurred in the weeks leading up to the start of the semester and ended on the final 
registration day for classes. 

The research team collected a variety of data on program and control group members, 
including transcripts and financial aid records from the two colleges in the study and informa- 
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tion on college enrollment and completion from the StudentTracker Service of the National 
Student Clearinghouse (a repository of information from colleges and universities throughout 
the United States). In addition, program and control group members were asked to complete a 
follow-up survey on college experiences, social interactions, future outlook, and health. 

In addition to the devastating effects it had on the people and communities of New Or- 
leans, Hurricane Katrina posed some challenging problems for the evaluation, especially the 
follow-up survey. At the time the hurricane stmck, close to half the sample had already com- 
pleted the survey. Since 60 percent of sample members resided within the city limits of New 
Orleans — which was submerged under as much as 16 feet of water — most sample members 
relocated to nearby regions and states after the storm. The survey effort was suspended for sev- 
eral months in the wake of the storm and later resumed. Ultimately, 79 percent of sample mem- 
bers were located and completed an interview. As noted earlier, the average follow-up period 
for the survey was about 16 months after random assignment, although the response times 
varied depending on whether students completed the survey before or after the hurricane. An 
appendix to this report describes the survey issues in greater detail and presents separate find- 
ings for pre- and post-hurricane respondents. 



Did the Program Make a DifFerence? 

This report presents findings on a number of outcomes, including educational attain- 
ment, level of effort, and health and well-being. Some of these outcomes were expected to be 
influenced immediately by the program. For example, it was hypothesized that registration and 
short-term academic achievement would increase. Other changes were expected to take place 
over the longer term, such as changes in health. These expectations were largely met by the 
program, and, in many cases, the findings exceeded expectations. 

As discussed above, some analyses of the effect of the program must account for the in- 
termption by Hurricane Katrina. This is accomplished by looking at the periods prior to and af- 
ter the hurricane for some outcomes and using measures from the survey that were asked of a 
majority of respondents. These approaches provide some insight into what would have hap- 
pened had Katrina not occurred. 

This report finds: 

• The Opening Doors program encouraged greater registration and full- 
time enrollment during the first and second semesters. 

Transcript data suggest that program group students were more likely, by 5.3 percentage 
points, than control group students to register during the first semester, and more likely, by 6.4 
percentage points, to register for 12 or more credits (full time). During the second semester. 
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program group students were more likely to persist with their studies, by 15 percentage points. 
This represents an increase in second semester registration of slightly more than 30 percent. The 
Opening Doors program also affected academic success. Program group students were more 
likely, by about 11 percentage points, to earn a 2.0 GPA than were control group students. 
These findings for the full sample are positive (and statistically significant), and they suggest 
that the program encouraged academic success, but Hurricane Katrina makes it impossible to 
know what would have happened after two semesters for the full sample. 

• The study found evidence of impacts on registration lasting through the 
fourth semester after random assignment — two semesters after the 
program largely ended — for the first two cohorts, or groups of stu- 
dents, in the program. 

The first two cohorts to participate in the program have data available for up to four 
semesters prior to Hurricane Katrina, permitting a look at trends that extend beyond the scholar- 
ship semesters. Figure ES.l shows that program group students in the first two cohorts were 
18.1 percentage points more likely, on average, than control group students to register in the 
second semester. In addition, program group students were 1 1.8 percentage points more likely 
than control group students to register in the third semester, which represents a one-year period. 
Registration rates continue to be higher for the program group in the fourth semester, the latest 
semester available for both groups before Hurricane Katrina. Note that despite the gains by the 
program group, both groups show some attenuation over time, with overall levels of registration 
declining somewhat each subsequent semester. 

• The Opening Doors program resulted in a larger number of credits 
earned for program group members. 

Figure ES.2 turns to the number of credits earned for students in the first two cohorts. 
Program group students earned more credits than control group students in all semesters except 
the fourth semester, at which point the difference was not statistically significant. The last two 
sets of bars in the figure show cumulative gains in credit acquisition for semesters one through 
four and one through seven. The program group earned 3.5 credits more than control group stu- 
dents over the first four semesters. While the colleges closed for the fall 2005 semester after 
Hurricane Katrina, some students managed to take online courses and register at other campuses 
that opened the following spring. The last set of bars shows that few credits were earned after 
the hurricane, with the result that the total cumulative credits earned from semesters one through 
seven were higher for program group students, who earned close to four credits more than con- 
trol group students. 
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The Opening Doors Demonstration 
Figure ES.l 

Key Impacts on Registration: First Two Cohorts 
Delgado Community College and Louisiana Technical College Report 




Semester after random assignment 

SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College 
transcript data. 

NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



• Opening Doors was designed to supplement the Pell Grant and other fi- 
nancial aid programs. Not counting the Opening Doors scholarship, 
program and control group members received virtually identical finan- 
cial aid awards in the first semester after random assignment. In the 
second semester, however, more program group members than control 
group members received financial aid awards, owing mainly to the fact 
that they were more likely to be registered for school. 

The Opening Doors scholarship was available only to members of the program group. 

Both program and control group members were potentially eligible for federal Pell Grants and 
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The Opening Doors Demonstration 
Figure ES.2 

Key Impacts on Credits Earned: First Two Cohorts 
Delgado Community College and Louisiana Technical College Report 
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Semester after random assignment 



SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College 
transcript data. 

NOTE: Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 



other financial aid, but those programs were administered separately from the Opening Doors 
scholarship. (As noted elsewhere in this report, the Opening Doors scholarship was designed as 
a supplement to existing financial aid, not a replacement for it.) In practical terms, this meant 
that the Opening Doors scholarship should increase the amount of money available to students 
in the program group. Importantly, the money from the Opening Doors scholarship was paid 
directly to the student, and could be used for school expenses along with basic living expenses 
and other needs. This was also very different from Pell Grants, which went directly to the insti- 
tution to cover tuition and fees, and may not have felt as tangible to students. 
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In the first semester after students were randomly assigned, a slightly higher percentage 
of program group members registered for classes than control group members (85 versus 80 
percent), but students in the program and control groups were awarded financial aid at virtually 
identical rates. This was an expected result. Students typically apply for financial aid several 
weeks (if not months) before the start of the semester, and the staff from the Opening Doors 
program did not have any influence on students’ decisions to apply or whether they were 
awarded aid packages. In the second semester after students were randomly assigned, more 
program group members than control group members (69 percent versus 56 percent) made the 
decision to re-enroll in college, and thus were also more likely to be awarded financial aid. 

• Students in the Opening Doors program are more likely than those in 
the control group to report that they are pursuing personal goals, have 
higher levels of self-esteem, and have more social support. 

At the start of the study, students exhibited relatively high levels of psychological dis- 
tress, suggesting that there might be room for significant impacts on their social and psycholog- 
ical well-being over time. After two semesters of the intervention, analyses based on the follow- 
up survey indicated that the Opening Doors program had positive impacts on a range of social 
and psychological outcomes, promoting greater levels of engagement in the pursuit of personal 
goals and in life more generally. Moreover, the study suggests positive program effects on two 
key aspects of identity: self-esteem and overall sense of self The latter includes a student’s 
sense of self both as an individual and in relation to significant others. Finally, the program ap- 
pears to have engendered a greater sense of perceived social support and contributed to more 
political engagement. All these findings suggest that the program enhanced the social and psy- 
chological well-being of participating students. 



What Are the Implications of the Results? 

The Opening Doors program in Louisiana owes its success to a number of fac- 
tors, including the following: 

• Provision of additional funds to students 

• Focus on a particularly receptive group of nontraditional students (largely 
single parents) 

• Transference of funds directly to students 

• Incentive nature of the scholarship 

• Provision of additional support through regular contact with counselors 

• Operation in a state that largely does not provide a generous need-based 
state-level financial aid program 
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The Opening Doors program in Louisiana targeted those with income beneath 200 per- 
cent of the federal poverty level and provided funds that were paid in addition to other financial 
aid for which the students had already qualified. In other words, the funds represented a net gain 
in aid. Further, the study sample was largely made up of nontraditional students (specifically, 
single parents) who are often penalized in financial aid calculations, which assume that the pre- 
vious year’s income will be available to support the student’s education the following year. In 
addition, the performance-based scholarship was distributed directly to students, which may 
have allowed them to address the financial needs, such as child care expenses, that would have 
most prevented their attendance and success. Finally, the program operated in a state that does 
not have a generous need-based state-level grant program for undergraduates, which may have 
made the students in the program group particularly receptive to the Opening Doors program, 
since the amount of aid provided in Louisiana is limited. 

Overall, the Opening Doors program resulted in an education gain of one-third of a 
semester, or about four credits. While four credits may not seem to be a large gain in terms of 
what is required to earn an associate’s degree, such a gain, taken in context, is notable. First, the 
students in this study face a multitude of barriers, most working while attending school and all 
caring for children (the average age of the youngest child is 3 years), mostly as single mothers. 
Very low persistence rates have been found for such nontraditional students in other studies. 
Second, the overall gain moves students toward the “tipping point’’ of 20 credits, which some 
research suggests is associated with a higher likelihood of graduation.^ 

While the study’s findings are compelling, they leave a number of open questions. Sev- 
eral aspects of the program’s design may be critical for its success, but without studies of the var- 
iation in those elements, the relative importance of each one remains unknown. For example, the 
context for offering financial aid — which, in this study, is the relatively low amount of need- 
based aid offered to students in Louisiana — may play a very large role in the findings, or, alter- 
natively, the maturity of students may be primarily responsible for the positive findings, or fur- 
ther still, the counseling component of the study could be critical. Most important, it is not clear 
from the present study whether the positive findings would have continued over time or reversed; 
the impact of Hurricane Katrina and its aftermath on the state and on surrounding regions adds to 
the uncertainty. 



* * * 

Although this study of a program offering a performance-based scholarship with coun- 
seling in Louisiana has raised more questions than were originally posed, it has also produced 



^Clifford Adelman, Answers in the Tool Box: Academic Intensity, Attendance Patterns, and Bachelor’s 
Degree Attainment (Washington, DC: U.S. Department of Education, 1999). 
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some noteworthy results and appears to offer a promising intervention to increase students’ per- 
sistence in pursuing their postsecondary education. However, it is just one study in one place. 
MDRC has made it a priority to replicate the program (and variations of it) to build more evi- 
dence on the potential for performance-based scholarships to help at-risk students attending 
community colleges and nonselective universities (that is, universities with minimal acceptance 
requirements or open admissions) in other states. The goal of the Performance-Based Scholar- 
ship demonstration — MDRC’s replication demonstration launched in 2008 in California, New 
Mexico, New York City, and Ohio — is to evaluate whether such scholarships would be as suc- 
cessful in different geographical locations, for different target groups, with different amounts of 
monies, and over varying durations. 



ES-12 




Chapter 1 

Introduction 



This report presents the results from a rigorous study of an incentive scholarship pro- 
gram with a counseling component for low-income parents in two New Orleans-area communi- 
ty colleges. Operated from 2004 to 2005 as part of the multisite Opening Doors demonstration, 
the program provided students with scholarships of up to $1,000 for each of two semesters (or 
up to $2,000 per student, in total), paid in increments based on each student’s success in meet- 
ing key benchmarks: 

• $250 upon enrollment at least half-time (defined as six or more credit hours) 

• $250 after midterms, contingent on staying enrolled at least half-time and 
earning a “C” average or better 

• $500 upon completion of courses, with a “C” average or better across all 
courses (that is, not per class) 

Scholarships were paid in addition to other financial aid for which students were 
eligible. The program was designed to help the students cover more of their expenses and to 
provide a financial incentive to make good progress in school. If students failed to earn the 
second payment because of poor midterm grades, they could receive a $750 check (representing 
the combination of the second and third payments) at the end of the semester if they brought 
their grades up to a “C,” or 2.0 grade point average (GPA), by that time. While the incentive 
scholarship was available to students for up to two semesters, students did not have to enroll for 
two consecutive semesters to qualify. Rather, students could take a semester off and return later 
(at least until the program ceased operation). Students who failed to meet the full terms of the 
scholarship in the first semester were allowed a “fresh starf ’ in the second program semester. In 
other words, the “C” average that was required to receive the scholarship did not have to be 
earned consistently across consecutive semesters. Counselors who were specifically hired to 
work with Opening Doors students provided counseling that supplemented the services that 
were available to all students, and monitored grades. 

This report — a collaboration between MDRC and its partners in The Network on 
Transitions to Adulthood, funded by the MacArthur Foundation — presents information on how 
the Opening Doors program was implemented in the Louisiana colleges and its effects up to two 
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and a half years after students entered the study.' One of a series of reports presenting findings 
from the Opening Doors demonstration sites, it updates and supplements the early findings that 
were presented in a research brief in 2006.^ The 2006 early findings brief examined the first two 
groups of students (or cohorts) to enter the study and showed that the incentive scholarship 
program led to better academic outcomes, including more full-time enrollment, greater academic 
success (as measured by the number of courses passed and course credits earned), and higher 
rates of college registration in the second and third semesters after entering the study. 

This chapter begins with a brief overview of the Opening Doors demonstration and key 
questions the demonstration seeks to answer. It contains a brief discussion of financial aid and 
then focuses specifically on the study in Louisiana, describing the colleges and the Opening 
Doors program model. The impact of Hurricane Katrina is then discussed. The chapter con- 
cludes with a description of the contents of the remainder of this report. 



The Opening Doors Demonstration and Evaiuation: Why invest in 
Postsecondary Education? 

An investment in postsecondary education has been repeatedly shown to pay high mon- 
etary and nonmonetary dividends to both students and society at large. In fact, one of the clear- 
est labor market trends is that real wages have risen much more for workers with higher levels 
of education, resulting in a growing disparity in earnings between workers with and without 
postsecondary education.^ For example, in 2006 those with some college experience or an asso- 
ciate’s degree earned 12 percent more than high school graduates and 66 percent more than high 
school dropouts."' In addition, holders of associate’s degrees were about half as likely to be un- 
employed compared with high school dropouts and about one-fourth as likely to live in poverty 
compared with high school dropouts.^ Nonmonetary benefits have been noted as well, such as 



'Members of The Network on Transitions to Adulthood are Gordon L. Berlin (MDRC), Mark Courtney 
(University of Washington), Sheldon Danziger (University of Miehigan), Connie A. Flanagan (Pennsylvania 
State University), Frank F. Furstenberg (University of Pennsylvania), Vonnie C. MeLoyd (University of North 
Carolina, Chapel Flill), Wayne Osgood (Pennsylvania State University), Jean E. Rhodes (University of Massa- 
ehusetts, Boston), Ceeilia E. Rouse (Prineeton University), Ruben G. Rumbaut (University of California, Ir- 
vine), Riehard Settersten (Oregon State University), and Mary C. Waters (Harvard University). Christina Pax- 
son of Prineeton University and Allen LeBlane of San Franeiseo State University are leading the evaluation 
eomponent foeused on health outeomes. 

^For the first report in the series, see Serivener et al. (2008). See Broek and Riehburg-Hayes (2006) for the 
early findings of the Louisiana program. 

^Mishel, Bernstein, and Allegretto (2007). 

"'U.S. Census Bureau (2007), Table A-3. 

^Caleulations based on College Board (2007a), Figure 1.10 and Figure 1.11. Note that these differenees in 
earnings may not be entirely attributable to edueation level, but may be partially aeeounted for by a variety of 
other faetors that affeet earnings, sueh as soeioeeonomie status (Rouse, 2005). 
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better health outcomes, children who are better prepared academically, and greater civic partici- 
pation — benefits that accme to the individual students, their family, and society as a whole.'’ 

Community colleges play an important role in providing opportunities for a large swath 
of students to participate in such human capital investment. The open admissions policies and 
relatively low tuition and fees of these institutions enable millions of students, who may lack 
preparation or may otherwise be unable to afford college, to invest in their future. Yet, despite 
the accessibility and relative affordability of community colleges, many students who matricu- 
late at these institutions end their formal education prematurely. Longitudinal studies of post- 
secondary student populations indicate that 46 percent of those who begin their studies at com- 
munity colleges do not complete a degree or enroll elsewhere within a six-year time frame.’ 
Clearly, persistence and retention are challenges that community colleges must address in order 
to further help students reap the real dividends of postsecondary education. 

It may be even a greater challenge for community colleges to improve academic 
achievement and success among students who are older than traditional college age. One study 
of older undergraduates who work found that 62 percent of students who considered themselves 
workers first (and students second) had not completed a certificate or degree after six years and 
were no longer enrolled, compared with 39 percent of adults who described themselves as being 
students first (working only to cover minor expenses).* These low completion rates may be due 
to a number of barriers faced by low-income adults, including the characteristics of low-wage 
jobs (for example, absence of paid leave, lack of flexible work hours, unpredictability of hours 
or shift work), academic underpreparedness, and family obligations.'* Most low-income students 
have substantial unmet needs, and low-income parents tend to have among the highest unmet 
needs given their financial aid status as independent students with dependents. This is because 
the federal aid calculation for independent students assumes that their previous year’s income 
will remain available throughout their course of study despite college enrollment, and it as- 
sumes that students can afford to contribute heavily to support their investments in education. 
These assumptions may be reasonable for young students residing with their parents (tradition- 
ally aged dependent students), who hold summer or part-time jobs, but they are unreasonable 
for nontraditional students such as parents or other older adults. 



^See Kolata (2007) for a review of the literature on the relationship between education and health. See 
College Board (2007a), Figure 1.16a, for evidence on academic preparedness. See Dee (2004) on the relation- 
ship between education and civic participation. 

’U.S. Department of Education (2002). 

*Berker, Flom, and Carroll (2003). 

^Levin-Epstein (2007); Matus-Grossman and Gooden (2001); Golonka and Matus-Grossman (2001). 
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Overview of the Opening Doors Demonstration 

Building on previous efforts to learn about factors that affect community college stu- 
dents’ enrollment in school and completion of their studies, MDRC, with support from a consor- 
tium of funders, launched the Opening Doors demonstration in 2003. A review of prior research 
and focus groups with past, current, and potential community college students revealed some key 
factors that hinder students’ progress: underpreparation for college-level studies; the challenges 
of juggling school, work, and family; and institutional barriers, such as inadequate support ser- 
vices and insufficient financial aid.'° Opening Doors is testing three promising strategies that col- 
leges could adopt to address those factors and increase student achievement and persistence: 

• Curricular and instructional innovations, including “learning communi- 
ties” where students take blocks of classes with the same group of peers, cus- 
tomized instmctional support (provided by counselors), academic instmction 
for students on academic probation, and enhanced orientation courses to help 
students navigate the college experience 

• Enhanced student services, including stronger, more personalized academic 
advisement, career counseling, and tutoring (the latter provided by faculty or 
other students) 

• Supplementary financial aid, such as an incentive scholarship or money di- 
rected to specific education-related costs, like vouchers for textbooks 

After a nationwide search, MDRC selected six community colleges in four states to par- 
ticipate in the demonstration and evaluation. MDRC worked with each college to develop an 
Opening Doors program that combined two or three of the strategies listed above. Table 1.1 lists 
the participating colleges and provides a brief summary of their Opening Doors programs and 
the students who were served. 

The Evaluation 

The evaluation of the Opening Doors demonstration consists of multiple components. 
The quantitative analyses use data from a random assignment design. Random assignment, or 
experimentation, is a methodology that assigns program participants (in this case, students) to a 
program condition (such as scholarships) or a control condition (that represents the status quo) 
randomly. That is, participation in the program is determined completely by a lottery as opposed 
to other mechanisms for determining participation such as “first come, first served.” Random 



*°See Matus-Grossman and Gooden (2001) for a discussion of the focus group results. 
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The Opening Doors Demonstration 
Table 1.1 

Opening Doors Programs and Target Groups 



Delgado Community College and Louisiana Technical College Report 



Site and MDRC Report 


Brief Description of Program 


Target Group 


Chaffey College 

Rancho Cucamonga, California 

Scrivener and Sommo 
(forthcoming) 


College survival skills and enhanced 
student services: Students took a one- 
semester guidance course that 
provided instructional support as well 
as advising; students were required to 
visit the college’s Success Centers, 
which provided extra academic 
support. 


Students ages 18-34 on 
academic probation who 
earned fewer than 35 credits 
and who either had a 
cumulative grade point 
average below 2.0 (C) or did 
not complete at least half the 
courses in which they enrolled 


Delgado Community College and 
Louisiana Technical College- 
West Jefferson 

New Orleans area, Louisiana 

Brock and Richburg-Hayes (2006) 


A scholarship predicated on 
academic performance, combined 
with counseling: Students were 
eligible for a $1,000 scholarship for 
each of two semesters ($2,000 total); 
scholarship was tied to maintaining at 
least half-time enrollment and a grade 
point average of 2.0 (C). In addition, 
students had access to an Opening 
Doors counselor with whom they were 
expected to meet periodically. 


Parents ages 18-34 whose 
family income was below 200 
percent of the federal poverty 
level 


Kingsborough Community 
College 

Brooklyn, New York 

Bloom and Sommo (2005) 
Scrivener et al. (2008) 


Learning communities and a book 
voucher: Groups of students took 
three linked courses together; students 
received enhanced advising and 
tutoring; students received vouchers to 
pay for textbooks. 


Incoming freshmen ages 17-34 
who planned to attend college 
full time 


Lorain County Community 
College 

Elyria, Ohio, and 

Owens Community College 

Toledo, Ohio 

Scrivener and Au (2007) 
Scrivener and Pih (2007) 


Enhanced student services and a 
modest scholarship: Students were 
assigned to an Opening Doors adviser 
with a small caseload with whom they 
were expected to meet frequently; 
students had access to designated 
contact in financial aid office; students 
were eligible for $150 scholarship for 
each of two semesters, paid after 
mandatory meetings with adviser. 


Students ages 18-34 whose 
family income was below 250 
percent of the federal poverty 
level and who either were 
incoming freshmen or had 
completed fewer than 13 
credits and had a history of 
academic difficulties 



SOURCE: MDRC field research data. 
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assignment is the “gold standard” of research because it holds everything constant except the 
program of interest. In this way, experimental designs permit the causal interpretation of differ- 
ences (or impacts) in outcomes between program group members and control group members. 

Findings from the evaluation are intended to yield information about how to improve 
the rate of student success in community colleges and will speak to the focus of the U.S. De- 
partment of Education’s renewed interest in the need for investments that lead to better student 
outcomes." 

Key Research Questions 

The Opening Doors evaluation includes three main components: an implementation 
analysis, an impact analysis, and a cost analysis. The key research questions in the implementa- 
tion analysis are: 

• Were the enhanced services sufficiently distinct from the services available 
to the study’s control group to constitute a “fair test” of the intervention? 

• What services were provided as part of the Opening Doors programs? What 
was their quality and intensity, and how might the programs help students? 

• How were the programs managed and operated, and how were the services 
delivered? 

The pathway through which the interventions are expected to affect outcomes is shown 
in Figure 1.1. The model provides a framework for linking the Opening Doors reforms to vari- 
ous outcomes that are important for a successful educational experience and transition to a bet- 
ter life. As illustrated in the figure, the Opening Doors reforms in curriculum and instmction, 
student services, and financial aid theoretically affect some key “early” educational or academic 
outcomes while the program is operating (such as the number of credits completed and academ- 
ic performance), as well as some “later” educational outcomes, including semester-to-semester 
persistence, graduation, and transfer to another postsecondary institution. 

As Figure 1.1 illustrates, the effects on the educational outcomes, if they are positive 
and strong, would in turn lead to improved labor market outcomes in the longer term, including 
better jobs, higher earnings, and enhanced well-being. Some of the social, psychological, and 
health outcomes (such as social networks or sense of self) might be affected in the short term, 
while others (such as physical and mental health) are more likely to be affected only in the 
longer term. 



"U.S. Department of Education (2006). 
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The Opening Doors Demonstration 
Figure 1,1 

Basic Conceptual Model for Evaluating the Effects of Opening Doors Programs 
Delgado Community College and Louisiana Technical College Report 
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As discussed above, the random assignment design provides a rigorous way to estimate 
the effects of the enhanced programs on various outcomes of interest. Reflecting the outcomes 
shown in Figure 1.1, the key research questions in the analysis of the program’s effects, or im- 
pacts, are: 

• Do the Opening Doors enhancements in curriculum, student services, and fi- 
nancial aid in community colleges lead to more positive early educational 
outcomes, including completing more credits and earning better grades, 
compared with standard college courses and services? (Each study measures 
the early educational outcomes only at the Opening Doors college, not at 
other postsecondary institutions.) 

• Do the enhancements lead to more positive educational outcomes at a later 
time, including higher rates of persistence in school, degree attainment, and 
transfer to four-year institutions? 

• Do the enhancements have a positive impact on students’ personal develop- 
ment, social networks, civic participation, or health behaviors? 

• Do the enhancements affect students’ success in the labor market? 

In each site in the study, MDRC and its research partners collected a range of data to 
help answer the impact and implementation research questions, including student transcript data 
and a survey that was administered about a year after students entered the study. Chapter 2 pro- 
vides more information about the data sources used in the analyses in this report. In addition, 
MDRC has collected data on the costs of operating each of the Opening Doors programs, as 
well as on the costs of providing each college’s standard classes and services. A future report 
will discuss the costs of the programs. 

A Limitation of the Research Design 

While a random assignment evaluation is considered the most reliable way to test pro- 
grams such as those in Opening Doors, this study’s design has an important limitation. The fo- 
cus groups with students and the other research that MDRC conducted when designing Opening 
Doors pointed to the need for programs with multiple enhancements and supports. To meet the 
identified needs, while ensuring that the program interventions were as robust as possible and 
clearly different from the colleges’ standard offerings, each college operated a program with at 
least two different components. As a result, the study can determine whether that package of 
enhancements led to outcomes that differed from those associated with the standard college 
classes and services, but the evaluation cannot disentangle the effects of each component. 




In the two Louisiana colleges, then, the study was intended to determine whether the 
package of incentive scholarships and counseling lead to different student outcomes compared 
with the outcomes that might be seen with standard financial aid and counseling services. It 
cannot, however, determine the specific effect of the counseling services separately from the 
incentive scholarships. The implementation research at each college sheds light on which pro- 
gram dimension may matter the most, but it does not yield definitive answers. That question 
will need to be addressed in future studies. 



Overview of Financial Aid 

The incentive scholarship component of the Louisiana intervention is similar to a finan- 
cial aid grant that is contingent on performance. One of the original purposes of student finan- 
cial aid was to ensure fairer access to postsecondary education to those least able to afford it and 
to those traditionally underrepresented. Various federal and state programs were put in place to 
achieve this goal, including the Pell Grant and the work-study programs. Although the Pell 
Grant program is the cornerstone of the federal student aid program, it continues to leave many 
students with substantial unmet need. For example, a single adult in Louisiana with two children 
with income at the poverty threshold in 2004 (income of $15,219) would receive the full 
amount of the Pell Grant ($4,050 in the 2004-2005 school year). However, at 200 percent of the 
federal poverty level (or income of $30,438), the adult student would be expected to contribute 
$2,273 toward her education and could expect a Pell Grant of about $1,800.'^ Given the total 
cost of attendance of $12,126 per year in 2004-2005 at Delgado Community College, this 
would result in an unmet need of more than $10,000.'^ 

In general, the trends in student aid have shifted from need-based aid toward merit aid 
and loans. This is evident in the increasing number of states operating merit scholarship 
programs. While the intents of merit aid (to keep the best and brightest students within their 
home state and help middle-income families with the rising cost of college) are admirable, such 
aid may have two possibly deleterious effects. First, merit scholarships do not equalize access. 
Based on high school records or standardized test scores, such scholarships may reward 
students with more privileged backgrounds and higher socioeconomic status since these 
characteristics are positively associated with better schools and better academic preparation. 
Second, merit scholarships may crowd out state-level aid based only on need. Trends in state 



*^See Long (2007) for similar calculations. Source: www.finaid.org/calculators/faaefc.phtml. Accessed 
February 17, 2008. 

*^MDRC calculations using cost of attendance data from the Integrated Postsecondary Education Data 
System (IPEDS). The total cost of attendance includes tuition, fees, books, and room and board. 

'“'Heller and Marin (2002). 
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financial aid show some declines or smaller growth in need-based aid awarded over the last 
decade in states that have increased merit aid awards.'^ 

How Does Financial Aid and Counseling Relate to Academic Success 
and Persistence? 

While many studies of the relationship between financial aid and college access have 
been done, few studies have looked at the relationship between financial aid and academic suc- 
cess. Questions about persistence in relation to financial aid have been difficult to answer be- 
cause of a lack of longitudinal data and problems with endogeneity. That is, factors that are as- 
sociated with financial need (such as low income) are also associated with a lack of academic 
success, so it is difficult to isolate the effect of additional financial aid on student achievement. 
However, the analyses that have been conducted suggest a positive relationship between grant 
aid and persistence.'® 

Similarly, there are few rigorous studies of the effect of advising/counseling and fewer 
studies of models that combine counseling with scholarships.'^ While academic guidance and 
counseling may arguably be the most important student service, most community college 
students receive minimal help as a result of the small number of counselors.'* Counseling, when 
available, often focuses on the basics, such as explaining terms like “course credif’ and 
“matriculation” to students, providing help with creating a class schedule, and so forth. 



The Opening Doors Program in Louisiana 

The Opening Doors program operated at the City Park and West Bank campuses of 
Delgado Community College (Delgado) as well as at the Louisiana Technical College-West 
Jefferson (LTC) campus. Starting in 2001, Louisiana began allocating funds from its Temporary 
Assistance for Needy Families (TANF) program (the cash welfare program that mainly serves 
single mothers and their children) to help low-income parents pursue higher education through 



'^National Association of State Student Grant and Aid Programs, (2007), Tables 4 and 5. For example, 
need-based aid decreased 78 percent over the 10 years from 1995 to 2005, while merit aid increased 1,276 per- 
cent over the same period. 

'®Bettinger (2004); Choy (2002); Leslie and Brinkman (1987). 

"'See Scrivener and Sommo (forthcoming) for findings on a student success and advisory intervention. 
Scrivener and An (2007) and Scrivener and Pih (2007) find modest effects of a counseling program with a 
small scholarship. Angrist, Lang, and Oreopoulos (2006) find small effects of a counseling and scholarship 
program for female students. 

'*Grubb (2001). 

'■'King (2002), p. 2. 
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the state’s community and technical college system.^'* MDRC and Louisiana officials began 
working together in 2002 to design what became the state’s Opening Doors program. The es- 
sential concept was to provide cash scholarships that would be paid in increments to provide a 
strong incentive for students to enroll at least half time while helping as many students as possi- 
ble earn the full scholarship. Scholarships were paid as supplements to other financial aid, such 
as the Pell Grant, not as a replacement form of aid. Program counselors, specifically hired to 
work with program group students, would monitor students’ academic performance and help 
resolve problems that interfered with school. Given the cost of establishing such a program and 
the uncertainty over whether it would lead to positive results, state officials decided to pilot the 
Opening Doors program at the two colleges discussed above. The Louisiana Department of So- 
cial Services and the Louisiana Workforce Commission agreed to fund and oversee the pro- 
gram, while MDRC assisted the colleges with program implementation and conducted all eval- 
uation activities, including random assignment of students to program and control groups. 



Effect of Hurricane Katrina 

The Opening Doors program in Louisiana was launched in spring 2004, and the last 
cohort planned to enroll in the spring of 2005. Hurricane Katrina, a category five hurricane, 
hit the Gulf Coast in August 2005, during the follow-up period of the study. In addition to its 
effects on the people and communities of New Orleans, this devastating natural disaster inter- 
mpted the study. 

Prior to Hurricane Katrina, the City Park campus of Delgado served more than 11,000 
students — among its all-time-high enrollment. The devastation from the storm was widespread 
and the areas where most sample members resided were among the hardest hit. Figure 1.2 
shows the flood depth levels of New Orleans and the prior residences of sample members. As 
the map shows, a substantial number of sample members lived in the hardest hit areas of New 
Orleans. Many New Orleans residents who did not evacuate the city were stranded in second 
stories or on rooftops until rescued by boat, a few as late as four or five days after the storm. 
Figure 1 .3 illustrates the effect of the hurricane on the enrollment at the two Delgado campuses 
studied in this report. 

As the figure suggests, the impact of Hurricane Katrina on the colleges in this study was 
devastating as well. At City Park, only three buildings were deemed salvable while nine other 
buildings — some of which were submerged in up to 9 feet of water — had to be razed. These 



^°Like many other states, Louisiana amassed a large funding surplus in its TANF program in the wake of 
rapid deelines in welfare easeloads. The federal government permitted states to use TANF funds in many ways, 
as long as they were used to help low-ineome families aehieve self-suffieieney. 
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Residences of Sample Members at Baseline and Flood Depths of Hurricane Katrina 
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NOTE: DCC = Delgado Community College; ETC = Louisiana Technical College. 






The Opening Doors Demonstration 
Figure 1.3 

Enrollment Patterns at Delgado Community College, by Campus 
Delgado Community College and Louisiana Technical College Report 




♦ City Park campus 
- ■ ■■ - West Bank campus 



Semester 



SOURCES: Delgado Community College Preliminary Enrollment Reports from City Park and West Bank 
Campuses, Fall 2007. 

NOTE: Fall 2005 semester was canceled due to Elurricane Katrina. 

buildings contained the administrative office, including the office that held hard copies of stu- 
dent transcripts, and the information technology office, among other important offices. National 
Guardsmen commandeered the West Bank campus shortly after the levees broke around the 
city. Despite the horrific circumstances, the college leadership was able to salvage backup cop- 
ies of student records and offer online courses to displaced Delgado students free of charge in 
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the weeks following the stom.^' Estimates suggest that approximately 650,000 people were 
displaced as a result of the storm.^^ 

Neither the city of New Orleans nor the colleges in this study have fully recovered as of 
this writing. In a survey of New Orleans residents who lived through the storm, 40 percent re- 
sponded that their lives are still very or somewhat dismpted three years after Hurricane Katrina. 
Nearly two-thirds reported that “good jobs are difficult to find” and 60 percent said that the re- 
building of New Orleans was not a priority for Congress and President Bush. More disconcert- 
ing, respondents reported an increase in serious mental illness between two years and three 
years after the storm. A corresponding number reported an increase in the use of prescription 
medication for problems with mental health.^^ Clearly, the effect of Hurricane Katrina compli- 
cates the interpretation of the findings after its occurrence. This report attempts to disentangle 
the effect of Katrina from the effects of the Opening Doors program on academic outcomes. 
Other work will continue to look at the ability of the sample members to cope in the face of a 
natural disaster. 



Contents of the Remainder of this Report 

The next chapter describes the characteristics of the sample members in the study and 
the various data sources used in this report. 

Chapter 3 provides further information about Louisiana’s Opening Doors program and 
discusses its implementation. 

Chapter 4 presents the program’s effects on various educational outcomes. 

Chapter 5 provides information on the program’s effects on selected social, psychologi- 
cal, and health outcomes. 

Chapter 6 presents some policy implications of the findings, discusses several important 
questions raised by this report, and outlines future research in this area. 



^’Delgado Community College Public Relations Department (2006). See Lederman (2005) and Evelyn 
(2005) for more information on the aftermath of the storm. 

^^U.S. Department of Commerce (2006). 

^^Kaiser Family Foundation (2008). 
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Chapter 2 



Sample Intake, Sample Characteristics, 
and Data Sources 



The Opening Doors evaluation is using a random assignment research design to esti- 
mate the effects of each college’s program compared with the regular services that are available. 
This chapter describes how students became part of the research sample in Louisiana and 
presents some characteristics of the sample members. It also discusses the data sources used in 
this report and the follow-up periods for the impact analyses. Among the main findings: 

• The program targeted low-income parents attending two community colleges 
in the New Orleans area. Recmitment began prior to the spring 2004 semester 
and students were enrolled for four consecutive terms, including the summer. 

• Opening Doors was a voluntary program and had to be “sold” to potential 
participants. Program staff conducted outreach on campus and (to a lesser ex- 
tent) in the community. The college orientation and testing sessions held for 
new students provided a main source of sample members; over time, word of 
mouth also helped recmitment. 

• The Opening Doors study enrolled a little more than 1,000 students in the re- 
search sample. The sample was predominantly female and black. Most sam- 
ple members were not married or living with a partner. On average, sample 
members had at least two children each. Most received food stamps or other 
government assistance. 



Target Population 

As noted in Chapter 1, Louisiana targeted the Opening Doors scholarship to low- 
income parents attending the City Park and West Bank campuses of Delgado Community Col- 
lege (Delgado) and Louisiana Technical College-West Jefferson (LTC). The program mainly 
targeted new students, but also accepted continuing students who were ready to make the transi- 
tion from developmental to college-level course work. To be eligible, students had to be willing 
to attend school at least half-time and meet the following criteria: 

• Be age 18 to 34 

• Be the parent of at least one dependent child under age 19 

• Have a family income below 200 percent of the federal poverty level 
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• Earned a high school diploma, General Educational Development (GED) 
certificate, or a passing score on a college entrance examination' 

• Not have a degree or occupational certificate from an accredited college or 
university 

These eligibility criteria were mutually agreed upon by the State of Louisiana and 
MDRC. Because program funding came through the Temporary Assistance for Needy Families 
(TANF) program, the state was particularly concerned about targeting adults who might either 
be receiving public assistance or be at risk of needing such assistance in the future. MDRC im- 
posed the age limit, mainly due to the interests of a few of the demonstration funders who 
wanted to learn about the effectiveness of programs designed to improve education and other 
outcomes among youth and young adults. 

The state required interested students to provide documentation that was similar to the 
required documentation for a welfare grant application. For example, students had to provide 
copies of their children’s birth certificates; show proof of income, including child support pay- 
ments, if applicable; and provide proof of a high school diploma, GED certificate, or entrance 
test scores from the college’s assessment office. Although it was not a condition for participa- 
tion in the study, the staff who handled intake also tried to make sure that all applicants had 
completed a Free Application for Federal Student Aid (FAFSA). The eligibility criteria for 
Opening Doors virtually ensured that everyone in the sample qualified for a federal Pell Grant, 
which would cover tuition and fees. (As noted in Chapter 1, the Opening Doors scholarship was 
intended as a supplement to Pell, not a replacement.) 



Recruitment and Random Assignment 

The recmitment and random assignment process used for the Opening Doors evaluation 
is depicted in Figure 2.1. In theory, the colleges could have identified eligible students from all 
those applying for financial aid, but decided instead to invite volunteers to participate in the 
study through a general marketing effort. This approach opened up the possibility of the 
scholarship to students who, for whatever reason, may not have applied for regular financial 
aid. The marketing and outreach campaign also gave college staff the opportunity to explain the 



*A previous publication by Brock and Richburg-Hayes (2006) on the Louisiana Opening Doors program, 
titled Paying for Persistence, stated incorrectly that students needed to have a high school diploma or GED 
certificate and a passing score on the college entrance examination. This wording gave some readers the mis- 
taken impression that the Opening Doors scholarship was only extended to students who were prepared to do 
college-level work. In fact, the entrance examination, also known as the “Ability to Benefif ’ test, covered pre- 
algebra number skills, writing skills, and reading skills. Its purpose was to determine students’ readiness for 
college and to place them in appropriate courses, which could include developmental courses. 
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The Opening Doors Demonstration 
Figure 2,1 

Recruitment and Random Assignment in the Opening Doors Evaluation 
Delgado Community College and Louisiana Technical College Report 
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program and the study to students, including the random assignment process, and to administer 
a brief baseline questionnaire for the study. Given the appeal of a $1,000 scholarship, neither 
MDRC nor the colleges expected recmitment to be very difficult or to take very long. 

The actual recmitment experience proved to be more challenging than first expected. 
Because the program was brand new, it took a while to establish a presence and become known 
on the campuses. Both Delgado and LTC set up the Opening Doors program in spaces that were 
physically removed from the admissions and financial aid offices, and that lacked natural “foot 
traffic.” Program staff quickly learned that they had to advertise the program and do aggressive 
outreach — tasks that most of them had never done before. During the first year of the study, 
MDRC staff worked closely with the colleges to hone their message and to develop effective 
strategies to meet the sample goal. 

The primary recmitment strategy adopted by Opening Doors was to seek program par- 
ticipants during orientation and testing sessions held for new students. Staff from the Opening 
Doors program set up recmitment tables at these sessions, passed out flyers to students standing 
in line, put up posters, and made oral presentations. They typically began by asking students 
whether they were parents and — if so — whether they needed money for college. Interested 
students were encouraged to visit the Opening Doors office. The chance of receiving an Open- 
ing Doors scholarship was the major selling point, but staff also emphasized that all eligible stu- 
dents who consented to be part of the study would receive a $20 gift certificate. 

Although staff mainly recmited students who had already stepped foot on the two cam- 
puses, they also did some outreach in the community — making presentations in local churches, 
for example. Staff got local radio stations to mn public service announcements about the pro- 
gram, and placed some advertisements in local papers. Over time, as the program became more 
established and the first group of students started receiving scholarship payments, word of 
mouth also helped bring students in the door. On balance, staff believed that most of the stu- 
dents who enrolled in the study had already decided to enroll in college and did not show up just 
because of the scholarship offer, although administrators at LTC credited the radio and newspa- 
per advertisements for increasing awareness of their campus in the community and boosting 
overall attendance at their institution. 

Recmitment for the study spanned four consecutive semesters, including the summer 
semester, with the first cohort recmited for the spring 2004 term and the last for the spring 2005 
term. Enrollment of sample members by cohort and campus is depicted in Figure 2.2. Not 
surprisingly, recmitment was most successful in the fall, when the new academic year began and 
the number of students on campus was the greatest. The smallest cohort entered during the second 
spring term, mainly because the program stopped taking applications after staff were confident 
they would reach their sample goal of 1,000 students. (In fact, they slightly exceeded their goal.) 
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The Opening Doors Demonstration 
Figure 2.2 

Enrollment of Sample Members, by Cohort 
Delgado Community College and Louisiana Technical College Report 




Spring 2004 Summer 2004 Fall 2004 Spring 2005 

Cohort 



SOURCES: MDRC calculations from random assignment data for Louisiana Technical College and 
Delgado Community College. 

During interviews conducted in the spring of 2005, staff from Delgado’s two campuses 
and LTC reflected on the recmitment process and some of the difficulties they encountered. 
Their major challenge was simply finding students who were both eligible for and interested in 
the program. With regard to eligibility, staff expressed particular frustration over the upper age 
limit (34), which they said excluded a significant number of low-income parents. Another chal- 
lenge was to convince students to believe the offer. A $1,000 scholarship stmck some students 
as too good to be tme, and they were wary of a “catch.” Finally, when Opening Doors began, an 
administrator at one of the campuses complained that the program was “thmst on him” and he 
did not have enough staff to do a serious job of recruitment, although this problem was later 
rectified. 

The program eligibility verification process — in particular, the requirement that appli- 
cants provide proof of income and parental status — deterred some students who either did not 
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have all the paperwork with them or felt it was too burdensome. Staff kept records of everyone 
who started the application process, and made phone calls or sent letters to students who were 
missing any forms, but some students never followed through. Staff reported that males, in par- 
ticular, sometimes balked at having to provide proof of paternity or custody of their children. In 
some cases, the fathers were not on amicable terms with ex-wives or girlfriends, making it 
awkward for them to get copies of papers that may have been in the mother’s possession, such 
as children’s birth certificates. 

Once students’ program eligibility had been determined, staff explained the random as- 
signment process, using the analogy of a lottery to describe how the scholarship recipients 
would be selected. They also emphasized that everyone who went through random assignment 
would receive a $20 gift card. Staff argued that students had nothing to lose and potentially 
much to gain by entering the study. No one could recall any students who refused to go through 
random assignment after getting this far into the process. 

Students who agreed to be part of the study signed an informed consent form, provided 
baseline demographic information, and completed a brief survey on health and other topics. 
Staff then telephoned MDRC, where random assignment was conducted. Staff informed stu- 
dents of their research status right away and gave them their gift cards. For students assigned to 
the program group, staff scheduled follow-up appointments with an Opening Doors counselor, 
usually within one week’s time, to get to know the students better and arrange the first scholar- 
ship payment. Staff told students who were assigned to the control group that they would not 
get a scholarship, but thanked them for being part of the study. If control group students had 
questions about where they could get counseling, child care, or other services, staff provided 
them with a list of other campus and community resources. 



Sample Characteristics 

A total of 1,019 low-income parents were randomly assigned to program and control 
groups in Louisiana. Delgado, as the larger of the two institutions in the study, accounted for 
roughly 80 percent of the sample; the remainder were at LTC. The characteristics of sample 
members at the time of random assignment, or baseline characteristics, are shown in Table 2.1. 
Because random assignment ensures that the characteristics of program and control group mem- 
bers are similar, the percentages below correspond to the full sample.^ 



comparison of the baseline characteristics of the program and control groups revealed a few items that 
had statistically significant differences, but no more than would be expected as a result of chance. See Appen- 
dix A for characteristics by program group status. An omnibus F-test indicated that the baseline characteristics 
as a set were not significantly different from one another. 
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The Opening Doors Demonstration 
Table 2,1 

Selected Characteristics of Research Sample at Baseline 



Delgado Community College and Louisiana Technical College Report 



Characteristic 


Full 

Sample 


Gender (%) 


Male 


7.6 


Female 


92.4 


Average age (years) 


25.3 


Age in years (%) 


17-18 


4.1 


19-20 


13.8 


21-34 


82.0 


Marital status (%) 


Married, living with spouse 


8.2 


Married, not living with spouse 


10.8 


Unmarried, living with partner 


6.4 


Unmarried, not living with partner 


74.7 


Race/ethnicity'* (%) 


Flispanic/Latino 


2.6 


Black, non-FIispanic 


84.9 


White, non-FIispanic 


10.5 


Asian or Pacific Islander 


0.4 


Multiracial 


0.7 


Other 


0.4 


Number of children (%) 


1 


50.9 


2 


27.2 


3 or more 


21.9 


Average age of youngest child (years) 


3.1 


Flousehold receiving any government benefits (%) 


70.9 


Unemployment/Dislocated Worker benefits 


4.4 


Supplemental Security Income (SSI) or disability benefits 


13.2 


Cash assistance or welfare (TANF) 


10.3 


Food stamps 


61.8 


Public or Section 8 housing 


18.0 


Financially dependent on parents (%) 


17.2 


Among those ever employed (%) 


97.7 


Average hourly wage at current or last job ($) 


7.27 



(continued) 
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Table 2,1 (continued) 



Full 

Characteristic Sample 



Among those employed at random assignment (%) 51.6 

Average hourly wage at random assignment ($) 7.34 

Diplomas/degrees earned'’ (%) 

Fligh school diploma 77.9 

General Educational Development (GED) certificate 18.6 

Occupational/technical certificate 9.8 

Date of high school graduation/GED certificate receipt (%) 

During the past year 10.7 

Between 1 and 5 years ago 31.6 

Between 5 and 10 years ago 33.2 

More than 10 years ago 24.5 

Main reason for enrolling in college'’ (%) 

To complete a certificate program 1 3 .5 

To obtain an associate's degree 56.3 

To transfer to a 4-year college/university 1 5 .5 

To obtain/update job skills 1 3 .3 

Other 6.1 

Completed/eamed any college courses/credits (%) 33.7 

First person in family to attend college (%) 42.6 

Working personal computer in home (%) 50.3 

Own or have access to a working car (%) 68.9 

Respondent bom in the United States’’ (%) 97.7 

Respondent or 1 or more parents bom outside the United States'’ (%) 4.6 



Sample size 1,019 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: Calculations for this table used all available data for the 1,019 sample members who completed a BIF. 

A two-tailed t-test was applied to differences between the research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and site. 

Characteristics shown in italic type are calculated for a proportion of the full sample. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

"Respondents who indicated that they are Hispanic and who also chose a race are included only in the 
Hispanic/Latino category. 

'’Distributions may not add to 100 percent because categories are not mutually exclusive. 

‘’"United States" includes Puerto Rico. 
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Over 90 percent of sample members were women, and more than four out of five were 
black. The average age of students in the sample was 25 years. The majority were not married 
and were not living with a partner. The sample was about evenly divided between parents who 
had one child and parents who had two or more. The youngest child in the household was just 
over 3 years old, on average. 

The vast majority of sample members had a high school diploma or GED certificate 
that they had earned several years prior to entering the study. The majority of sample members 
said their goal was to obtain an associate’s degree, and about one-third had already earned 
some college credits. Just over 40 percent said that they were the first in their families to attend 
college. 



Nearly everyone in the sample had some employment history, and about half were 
working at the time they entered the study. About 7 out of 10 sample members lived in a house- 
hold that received some government benefits — usually food stamps. Eighteen percent of the 
sample lived in public housing or received Section 8 vouchers, and 10 percent received cash 
assistance or welfare. Not surprisingly, given their age and parenting status, most sample mem- 
bers were financially independent from their own parents. 

Many of the sample members indicated that they had some resources available to them 
to help manage school and work obligations. For example, more than two-thirds said they owned 
or had access to a working automobile. In addition, just over half said they had a working per- 
sonal computer in their home. Almost all sample members were bom in the United States. 

Table 2.2 compares the characteristics of the Opening Doors sample at Delgado and 
ETC to first-time, first-year students at each of these institutions. (As noted above, about a third 
of sample members already had some college credits, but the majority were first-time students.) 
The comparison shows that Opening Doors sample members were more likely to be female and 
black than other first-time, first-year students, though they were similar in age. The comparison 
also shows that the Opening Doors sample was more likely to be receiving financial aid, at least 
at Delgado. (Information on financial aid receipt for ETC students in the research sample was not 
available.) In sum, the Opening Doors program targeted a particular segment of the student body 
and the results should not be generalized to all students who enrolled at these institutions. 
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The Opening Doors Demonstration 
Table 2.2 

Selected Characteristics of Colleges and Research Samples 



Delgado Community College and 
Louisiana Technical College-West Jefferson Report 





Delgado Community College” 


Louisiana Technical College- 
West Jefferson 


Characteristic (%) 


First-Time, 
First- Year 
Students 


Research 

Sample 


First-Time, 
First- Year 
Students 


Research 

Sample 


Gender 


Male 


33.8 


5.5 


49.5 


15.8 


Female 


66.2 


94.5 


50.5 


84.2 


Younger than 25 years'’ 


51.9 


50.2 


29.4 


32.7 


Race/ethnicity 


Hispanic 


2.9 


2.9 


5.3 


1.5 


White 


32.7 


10.1 


24.2 


11.7 


Black 


43.5 


84.8 


60.0 


85.2 


Native American 


0.7 


0.4 


3.2 


1.0 


Asian 


1.0 


0.4 


5.3 


0.5 


Other 


19.1 


1.4 


2.1 


0.0 


Financial aid. 2003-2004 


Federal grant aid received 


56.0 


80.7 


38.0 


NA 


Student loan aid received 


34.0 


64.6 


0.0 


NA 


Any financial aid received 


69.0 


81.9 


63.0 


NA 


Sample size 


2,680 


817 


95 


202 



SOURCES; U.S. Department of Education, National Center for Education Statistics, Integrated Postsecondary 
Education Data System (IPEDS); and MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: Data on first-time, first-year students are from fall 2004. Data on the research sample reflect the 1,019 
sample members who completed a BIF. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

NA = not available. 

“Delgado Community College data include the City Park, West Bank, and Charity School of Nursing 
campuses. Opening Doors operated at the main campus (City Park) and the West Bank campus. 

'’Data on age are based on the entire undergraduate student population. 
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Data Sources and Follow-Up Periods 

To examine the impacts and implementation of Louisiana’s Opening Doors program, 
the analyses presented in this report rely on several data sources, described below. 

Baseline Data 

As mentioned above, just before students were randomly assigned for the study, they 
completed a questionnaire, called the Baseline Information Form (BIF), and a baseline survey. 
The BIF collected demographic and other background information. The survey asked a series of 
questions about students’ social and psychological well-being and their health. Baseline data 
were used to describe the research sample, determine whether random assignment produced 
program and control groups with similar characteristics, and make statistical adjustments in the 
impact analysis. 

Transcript Data 

Delgado Community College and LTC provided to MDRC transcript data for the sam- 
ple members in the study. These data include various academic outcomes, including courses 
registered for, withdrawn from, and passed; number of credits earned; and grade point average 
(GPA). In this report, transcript outcomes are presented for up to seven semesters after random 
assignment. Transcript data are used in Chapter 4 to provide a detailed look at sample members’ 
performance in college. 

National Student Clearinghouse Data 

The National Student Clearinghouse, a nonprofit organization, collects and distributes 
enrollment, degree, and certificate data from more than 3,000 colleges that enroll more than 90 
percent of the nation’s college students.^ The Clearinghouse data are used in Chapter 4 to pro- 
vide important information about students in the study who may have attended a postsecondary 
institution other than Delgado or LTC. The Clearinghouse data cover up to seven semesters af- 
ter random assignment. 

Financial Aid Data and Scholarship Data 

From Delgado Community College, MDRC obtained records of all financial aid re- 
ceived by program and control group members from spring 2004 through spring 2007. The 
files include Opening Doors scholarships, federal Pell Grants, student loans, work-study 



^National Student Clearinghouse (2008). 
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payments, and any other scholarships and grants administered by the college. (From LTC, 
only data on Opening Doors scholarship payments were available.) Findings on Opening 
Doors scholarship payments are presented in Chapter 3 to describe program implementa- 
tion. Program impacts on all financial aid received from Delgado Community College are 
also presented in Chapter 3. 

The Opening Doors Follow-Up Survey 

MDRC and its research partners designed a follow-up survey to be administered to 
sample members approximately 12 months after random assignment. The survey contained 
questions on a wide range of topics, including sample members’ educational experiences, social 
relationships and supports, future outlook and identity, and health. Close to half the sample (492 
out of 1,019) completed the survey before Hurricane Katrina, with an average follow-up period 
of 13 months after random assignment. 

As a result of the hurricane, the follow-up survey had to be suspended for several 
months. MDRC and its research partners then revised the instmment to include new questions 
about adaptation and resiliency after the storm, and modified or dropped some items to be sensi- 
tive to the personal losses and dismption that sample members may have experienced during the 
storm. In April 2006, the revised survey went into the field, and interviews were completed with 
an additional 309 sample members. The fielding period for the revised survey lasted one year. 
On average, the post-Katrina respondents completed their interviews about 21 months after ran- 
dom assignment. 

Adding the pre- and post-hurricane respondents together, the survey sample consists of 
a total of 801 individuals, or 79 percent of the 1,019 students who were randomly assigned in 
the study. (See Figure 2.3.) The average follow-up period for the full survey sample is just un- 
der 16 months. Their responses are used in Chapters 3, 4, and 5. A discussion of the characteris- 
tics of respondents before and after Hurricane Katrina, and separate analyses of their responses 
to selected survey questions, are included in Appendix B. 

Qualitative Study of Women In the Opening Doors Sample 

As a complement to the survey administration after Hurricane Katrina, Mary Waters at 
Harvard University led a team of three researchers who conducted in-depth, qualitative inter- 
views in spring 2006 with a randomly selected group of 50 women from the Opening Doors 
sample. Both program and control group members were included. The women were chosen 
from the 492 respondents who had completed the follow-up survey before Hurricane Katrina, 
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The Opening Doors Demonstration 
Figure 2,3 

Follow-Up Survey Respondents Before and After Hurricane Katrina 
Delgado Community College and Louisiana Technical College Report 
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were located again after the storm, and expressed willingness to participate in a qualitative in- 
terview. The research team also selected respondents so that half were living in the New Or- 
leans area and half were living in the Houston area at the time of the interview. 

The qualitative interviews covered issues related to family life, educational goals and 
attainment, employment, and health, both before and after Hurricane Katrina. Particular atten- 
tion was given to the evacuation from New Orleans and subsequent resettlement. The inter- 
views lasted between one and three hours. All the interviews were conducted in person and 
were recorded, with the respondents’ permission. The interviews were subsequently transcribed 
and coded using a qualitative research software program. Findings from the qualitative inter- 
views are presented in Chapter 3. 

Field Research 

MDRC staff conducted interviews with college administrators, faculty, and staff on the 
operations of the Opening Doors program and on the key differences between the program and 
other services available to the members of the study’s control group. The majority of interviews 
were conducted in spring 2005, prior to Hurricane Katrina. In addition, MDRC staff kept notes 
from meetings when the program was first designed and launched, and from periodic program 
monitoring visits. Finally, MDRC staff collected college catalogs, program flyers, and other 
documentation pertaining to the marketing and operations of Opening Doors at Delgado Com- 
munity College and LTC. Information from the field research was used earlier in this chapter 
and is used again in Chapter 3. 
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Chapter 3 



Implementation of Opening Doors 
at the Louisiana Coiieges 



The Opening Doors program in Louisiana sounds simple enough: a $1,000 scholarship 
for each of two semesters ($2,000 total), offered on the condition that students stay enrolled at 
least half-time and maintain an average grade of “C” or better. The program also included some 
counseling. As with so many new initiatives, however, the devil lay in the details. How would 
staff determine whether students had “earned” the scholarship? How flexible would they be 
with students who had unusual circumstances? What would be the nature of the counseling? To 
what extent and how would students take advantage of the money and services offered? This 
chapter examines these and other issues to explain how the program worked in practice. Among 
the key findings: 

• The colleges and MDRC worked together to create procedures that would 
provide a strong incentive for students to stay enrolled in school and perform 
satisfactorily. Scholarships were paid out in installments — $250 at the start 
of the semester, $250 after midterms, and $500 after the end of the semester 
— so that students could demonstrate that they had met the conditions. 

• Ninety percent of students in Opening Doors received at least one installment 
from the scholarship. However, less than one-third of students received the 
full $2,000 over two semesters. 

• Program counselors were hired to meet with students periodically during the 
semester, monitor their academic performance, and hand them their checks. 

The meetings with students tended to be brief, and generally did not delve in- 
to academic or personal issues. Nevertheless, a follow-up survey indicated 
that students in the program group felt they got more support from the col- 
lege than did control group students. 

• Opening Doors was designed to supplement the Pell Grant and other finan- 
cial aid programs. Not counting the Opening Doors scholarship, program and 
control group members received virtually identical financial aid awards in 
the first semester after random assignment. In the second semester, however, 
more program group members than control group members received finan- 
cial aid awards, owing mainly to the fact that they were more likely to be reg- 
istered for school. 



29 




• During qualitative interviews, several women spoke positively about the ex- 
periences they had in Opening Doors. Both program and control group 
members expressed strong motivation to earn a college degree, though many 
also faced significant barriers. Not surprisingly. Hurricane Katrina dismpted 
the educational plans of most of the women who were interviewed, though 
many had taken steps to re-enroll in college. 

Overall, the findings suggest that Opening Doors received a “fair test.” That is, the key 
elements of the program — scholarships and counseling — were in place for the duration of the 
study and were clearly distinct from what control group members received. The results also 
highlight the difficulties many students experienced in making consistent academic progress, 
despite the Opening Doors intervention. 



The College Settings 

As noted in Chapter 2, about 80 percent of the students in the Opening Doors evalua- 
tion attended Delgado Community College.' At the time the study began, Delgado served 
15,000 students per semester on three campuses: City Park (the main campus). West Bank, and 
the Charity School of Nursing. Only City Park and West Bank participated in the study. The 57- 
acre City Park campus, with its stately brick buildings, broad lawns, and moss-covered oak 
trees, offered a distinctly collegiate atmosphere, whereas the 14-acre West Bank campus con- 
sisted of a cluster of attractive, modem buildings. Delgado offered 40 programs leading to asso- 
ciate’s degrees, with the largest enrollments in the health and business fields and the liberal arts 
and sciences. The college also offered 25 certificate programs in areas like motor vehicle main- 
tenance and repair, emergency medical technician, and child care. 

The remaining 20 percent of students in the Opening Doors study attended Louisiana 
Technical College-West Jefferson (LTC). One of the smallest postsecondary institutions in the 
state, LTC had an enrollment of just over 700 students when the study began, and its facilities 
were modest. LTC offered several occupational programs, with the most popular being mechan- 
ics, repair technologies, and nursing. Most of its programs led to occupational certificates, 
though associate’s degrees were offered in the health and business fields. 

Both Delgado and LTC served predominantly low-income populations. Tuition and 
fees at Delgado for a full-time student were about $1,500 in 2004-2005. That same year, about 
69 percent of Delgado students received some form of financial aid, most often in the form of 
Pell Grants (a federal program that awards aid to the neediest college students). Tuition and fees 



'information on the college settings is derived from the Integrated Postsecondary Education Data System 
(http://nces.ed.gov/ipeds) and the Delgado Community College 2004-2005 catalog. 
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for a full-time student at LTC in 2004-2005 were about $900. Similar to Delgado, 63 percent of 
LTC’s students received financial aid, also usually in the form of Pell Grants. (See Table 2.2 in 
the previous chapter for a breakdown of financial aid distributed at the two institutions.) 



Program Operations 

At both Delgado and LTC, Opening Doors operated as a small pilot program 
exclusively for the study. It began in spring semester 2004 and ended after the summer 2005 
term, just before Hurricane Katrina shuck. (Even if the hurricane had not occurred, no more 
funding had been allocated for the program.) Throughout the study period, each college 
appointed an adminishator to oversee Opening Doors and hired counselors to work with 
students. Delgado’s City Park campus, which served the most students, had two Opening Doors 
counselors, whereas the West Bank campus and LTC each had one. In addition. City Park and 
LTC hired an administrative assistant, who also served as a receptionist, for the counselors. 
Most of the administrators and staff held other student services positions at the colleges before 
being assigned to Opening Doors. 

Scholarship Policies and Procedures 

MDRC worked with Delgado and LTC staff to develop policies and procedures for the 
scholarship. As noted in the introductory chapter, it was usually paid in three installments: 

1 . $250 upon enrollment at least half-time (defined as six or more credit hours) 

2. $250 after midterms, contingent on staying enrolled at least half-time and re- 
ceiving passing grades at this point in the semester 

3. $500 on completion of six or more credit hours with a “C” average or better 

This offer was extended over two semesters, giving students the opportunity to receive 
up to $2,000 total. Students did not have to enroll for two consecutive semesters to qualify, but 
could take a semester off and return later. Because the program ended in August 2005, students 
in the first cohort had up to four semesters to use the scholarship, whereas students in the last 
cohort had to enroll in two back-to-back terms if they wanted to take full advantage of the offer. 

MDRC and the colleges wanted to create as strong an incentive as possible for students 
to take at least a half-time load and perform satisfactorily, and to receive all the money that was 
offered. Hence, if students did poorly at midterms and failed to earn the second payment, they 
could recoup the full amount if they completed six or more credits and brought their grades up 
to a “C” or better after final grades were posted. Similarly, if a student failed to meet the full 
terms of the scholarship in the first semester of the program, he or she still had the opportunity 
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to receive a $1,000 scholarship in the second semester. Notably, the “C” average required for 
the scholarship was based on each semester’s, rather than cumulative, performance. 

A couple of modifications in the scholarship procedures were made during the summer 
sessions at Delgado. (LTC operated the Opening Doors program the same way year-round.) 
Delgado’s summer classes lasted half as many months, but met for twice as many hours each 
session, as classes during the fall and spring semesters. Program staff felt that the summer 
semester was too short a period to monitor students’ progress and administer three separate 
payments, so they split the scholarship into two equal payments instead: the first made upon 
enrollment in six or more credit hours, and the second made upon satisfactory completion with 
a “C” or better average. In the last term that the program was in operation (summer 2005), Del- 
gado also permitted students to receive a half scholarship ($500 total) if they enrolled in a sin- 
gle, three-hour credit course. This change was made in response to staffs realization that many 
students could not manage more than one summer school course because of the longer class 
hours and other obligations for work or child care. Students who managed to take six or more 
credit hours and earn passing grades still received the full $1,000. 

The Counseling Role 

The Opening Doors counselors were responsible for interpreting the scholarship mles 
and getting the money into students’ hands. As the program was first conceived, MDRC hoped 
that the counselors would get to know students on a personal level, encourage them to stay in 
school and make good progress, and take an active role in making referrals to appropriate ser- 
vices on campus or in the community if academic or personal problems interfered with school. 
Some counselors came closer to fulfilling this vision than others, but as a group, the counselors 
mostly played a monitoring function. This meant that they checked up on students’ enrollment 
status and grades, and met with students periodically to explain program mles and give them 
their checks. Most of their meetings with students were kept under 30 minutes. 

At Delgado, counselors were usually able to perform the monitoring function by going 
onto the college’s online system, which included information on enrollment and grades. If for 
some reason grades were missing, the counselors contacted the faculty directly to ask how the 
students were doing. (Contacting faculty was the standard procedure at LTC, which lacked an 
online system.) As one counselor described, at least some Opening Doors students were aware 
that they were being monitored: 

I had one student call me Big Brother. I said no, I am a Big Sister. She said 
no, you’re like Big Brother in that TV show. My instmctor tells me you are 
calling to find out how I am doing and that he reports back to you. You’re 
watching so I feel like I have to do well. 
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The counselors did not try to influence the grading process. They were not allowed to 
tell an instructor, for example, that a student needed a passing grade to keep her scholarship. On 
occasion, however, counselors took action if there was a problem that they could help the stu- 
dent rectify. For example, sometimes counselors discovered that students were still enrolled in 
classes that the students thought they had dropped. In such instances, counselors explained to 
students what they needed to do to withdraw from the course formally. 

In addition to the monitoring, it was the counselor’s job to initiate the scholarship pay- 
ment process. Once counselors confirmed that students met the conditions for a scholarship 
payment, they notified the bursar’s office to request that a check be issued. When the checks 
were ready — typically within one week of the request date — the counselors picked up the 
checks and notified the students. The counselors took the checks back to their offices and per- 
sonally handed them to students, using this as an opportunity to ask students how things were 
going and to reinforce the sense of “reward” for good academic progress. 

During interviews with MDRC staff, the counselors indicated that they found their jobs 
to be manageable. Their caseloads ebbed and flowed depending on the number of participants 
each semester, but rarely exceeded 100. (The number of active participants was even smaller, 
since some students dropped out of college or enrolled less than half-time.) What the counselors 
liked most about their jobs were their interactions with students, although — as noted above — 
these interactions tended to be of short duration. The counselors did not try to offer advice on 
selecting courses or majors, nor did they delve into students’ personal relationships or psycho- 
logical issues. They generally were not trained to perform such functions, and their offices 
lacked the privacy that would be needed to broach sensitive topics. 

In interviews with MDRC staff, the counselors expressed strong feelings that they made 
a difference for students. One counselor extolled the program’s role in helping students take 
greater responsibility for communicating with their insttuctors; another noted the importance for 
every student of “having access to that one person who cares.” MDRC researchers tended to 
have less favorable impressions of the counseling role, mainly because it did not reflect an ac- 
tive effort to intervene with students who were having problems and to make referrals to other 
campus or community resources. One counselor, for example, communicated with students al- 
most exclusively via e-mail and did not appear to relate well either to students or to other staff 
on campus. Whatever the counselors’ limitations, however, they clearly offered more attention 
and advising to students in the program group than members of the control group received. 
(More information on this point is offered later in the chapter.) 
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Administrative Support for Opening Doors 

As noted in Chapter 2, the Opening Doors program functioned separately from finan- 
cial aid or regular counseling and advising services on the campuses. This gave the program a 
special feel, but also created a sense of isolation. Indeed, the Opening Doors staff did not feel 
that the program was well supported by the leadership at their institutions; one staff member 
went so far as to say that Opening Doors felt like the “ugly stepsister.” Top-level administra- 
tors at Delgado and LTC were more diplomatic, but generally confirmed that Opening Doors 
was not a top priority. The lack of leadership support may have reflected the fact that Opening 
Doors was perceived as a program that the State of Louisiana and MDRC brought to the col- 
leges, rather than one that the colleges themselves initiated. Opening Doors was also perceived 
as having a funding stream (state welfare surplus dollars) that would not last. One top adminis- 
trator made clear that his time and energy were better devoted to initiatives that would affect 
all faculty, staff, and students, rather than a small pilot program like Opening Doors. Fortu- 
nately, the leadership at the two colleges was sufficiently concerned about the evaluation to 
make sure that the program maintained adequate staffing and fulfilled its scholarship commit- 
ments to students throughout the study period. 



Participation in Opening Doors and Use of the Schoiarship 

To what extent did students actually participate in the program and earn the scholar- 
ship? As shown in the bottom panel of Table 3.1 (first line under “Summary”), 90 percent of 
students in Opening Doors received at least one scholarship payment after they were randomly 
assigned. Over two semesters, the average total scholarship received was $1,133. Slightly less 
than one-third of the students received the full $2,000. 

The top panels of Table 3.1 provide more insight into how students drew down the 
scholarship. (The first line of each panel shows the percentage of program group members who 
received one or more scholarship payments in a semester; the indented lines underneath pro- 
vide a breakdown of the actual amounts received.) It was expected that most students would 
receive the first $250 payment, since the only criterion was that they enroll at least half-time — 
a condition that was made explicit when they agreed to participate in the study. During the first 
semester after random assignment, 84 percent of the Opening Doors students received one or 
more scholarship payments. (The other 16 percent either did not enroll in college at all or 
enrolled less than half-time.) Sixty-one percent of those who received a scholarship earned the 
full $1,000; the remainder received lesser amounts, either because they did not maintain at 
least half-time enrollment or did not earn at least a “C” average in that semester. 

As noted earlier. Opening Doors students were not required to stay enrolled for two 
consecutive semesters, but Table 3.1 indicates that most of them did. Specifically, 62 percent of 
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The Opening Doors Demonstration 
Table 3.1 



Scholarship Receipt Among the Program Group 
Delgado Community College and Louisiana Technical College Report 

Program 

Outcome Group 

First semester after random assignment 

Received one or more scholarship payments (%) 84.0 

Received exactly $250 22.4 

Received exactly $500 15.1 

Received exactly $ 1 ,000 60.6 

Received other amount"* 1 .9 

Average scholarship amount received ($) 752 

Second semester after random assignment 

Received one or more scholarship payments (%) 62.2 

Received exactly $250 23.9 

Received exactly $500 14.6 

Received exactly $ 1 ,000 58.3 

Received other amount** 3.2 

Average scholarship amount received ($) 736 

Third through fifth semesters after random assignment 

Received one or more scholarship payments (%) 6.9 

Received exactly $250 37.1 

Received exactly $500 20.0 

Received exactly $ 1 ,000 40.0 

Received other amount"* 5.7 

Average scholarship amount received ($) 636 

Summary ** 

Received any scholarship payment (%) 89.7 

Received scholarship payments in 2 semesters 63.2 

Received scholarship payments in 2 consecutive semesters 60.0 

Received scholarship payments in 2 nonconsecutive semesters 3 .2 

Received the full $1,000 in at least 1 semester 59.4 

Received the full $2,000 over 2 semesters 30.5 

Average total scholarship payment received over 2 semesters ($) 1,133 

Sample size 505 

(continued) 
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Table 3.1 (continued) 

SOURCE: MDRC calculations from Opening Doors scholarship data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. Calculations for this table used all 
available data for the Spring 2004, Summer 2004, Fall 2004, and Spring 2005 cohorts. 

The scholarship was offered from spring 2004 through summer 2005. Depending on semester of random 
assignment, students had between two and five semesters to take up the scholarship. 

The outcomes showing exact amounts are based on scholarship recipients for that semester, rather than all 
program group members. 

^Three students at Louisiana Technical College received scholarship payments of $375, and 16 students at 
Delgado Community College received payments of $750. 

'’Summary outcomes include data for spring 2004, summer 2004, fall 2004, spring 2005, and summer 2005. 



Opening Doors students received a scholarship in the second semester after random assignment, 
whereas only 7 percent received a scholarship payment in the third, fourth, or fifth semester after 
random assignment. (Recall that the option to leave and come back to the program pertained to the 
first three cohorts only. The fourth cohort could participate in the spring and summer 2005 ses- 
sions only.) Similar to the pattern in the first semester after random assignment, 58 percent of stu- 
dents in the second semester after random assignment received the full $1,000. Students who re- 
ceived a payment in the third through fifth semesters after random assignment (suggesting that 
they took a break from the program and later returned) were less likely to receive the full $1,000. 

The patterns of scholarship receipt suggest two main conclusions. First, although the 
conditions of the scholarship (half-time enrollment and a “C” average) were not intended to be 
onerous, a sizable proportion of students were unable to meet them. Second, students’ ability to 
earn the scholarship did not improve over time. It may be that some students did not think the 
scholarship was worth the effort, though interviews with program administrators and staff did 
not lend support to that interpretation. In the words of one administrator: “Of course, the stipend 
made a difference... $250 makes a difference between someone having their utilities on and 
not. You need lights on in order to study. It makes a difference for everybody.” 

Program administrators and staff felt that three main factors explained why so many 
students failed to earn the full amount of money that was offered. One was weak academic 
preparation for college, which the scholarship did not address. A second was the difficulty some 
students faced in balancing school, work, and family responsibilities. A third included severe 
health problems or personal traumas, such as domestic violence, that interfered with school for 
some students. The counselors’ perceptions were reinforced by comments made by some of the 
women who participated in intensive qualitative interviews, discussed later in this chapter. 

In the Opening Doors follow-up survey, respondents who received a scholarship pay- 
ment were asked an open-ended question about how they used the money. As shown in Table 
3.2, their answers indicated that the scholarship mostly went toward basic expenses associated 
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The Opening Doors Demonstration 
Table 3.2 



Reported Use of Opening Doors Scholarship 
Delgado Community College and Louisiana Technical College Report 



Outcome'* (%) 


Sample 

Size 


Program 

Group 


Uses of scholarshiD** 


To purchase books and school supplies 


291 


65.6 


To help pay bills 


291 


45.7 


To buy gas or bus fare 


291 


45.4 


To help with child care costs 


291 


44.0 


To help with tuition and fees 


291 


22.3 


To buy food 


291 


21.0 


To buy clothes or shoes for children or other family members 


291 


18.9 


To buy clothes or shoes for myself 


291 


12.4 


For deposit in bank account 


291 


7.9 


For entertainment 


291 


1.7 


Other"* 


291 


4.8 


Main use of scholarshin"* 


To purchase books and school supplies 


289 


45.7 


To help with tuition and fees 


289 


26.3 


To help pay bills 


289 


16.6 



SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College transcript 
data and 12-month survey data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

“Table calculations based on 291 respondents to the 12-month follow-up survey who said they received an 
Opening Doors scholarship. 

'’Distributions may not add to 100 percent because categories are not mutually exclusive. 

Tncludes respondents who responded "I don't know" to the survey question "flow did you use this 
money? (Check all that apply)" or who responded "Other" and wrote in an answer. Other uses specified 
include buying a computer, buying a car, making car repairs, paying for medication, buying school clothing, 
equipment, or uniforms, and living off the money. 

‘'Only the top three uses are presented here. Two students did not report a main reason and are excluded 
from this sample. 



with going to school or maintaining their household. Two-thirds of respondents said they used 
the scholarship to purchase books and school supplies. Between 44 and 46 percent of students 
indicated that they used the money to help pay bills, buy gas or bus fare, or help with child care 
costs. When asked about the main use of the scholarship, a plurality of students said it was for 
school-related expenses. 
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The Contrast Between Opening Doors and Regular Services 

For a program to make a difference in students’ lives, it must not only be implemented 
well, but must also be distinct from other programs and services that are available to them. In 
the case of Opening Doors, for example, there would be little hope of detecting program effects 
if control group members received a similar incentive-based scholarship from another source, or 
if the Opening Doors scholarship resulted in a dollar-for-dollar reduction in financial aid that 
students would normally receive. This section compares the programs and services available to 
program and control group members in order to answer the question of whether Opening Doors 
was distinctive and received a “fair test.” 

Differences in Receipt of Financial Aid 

The Opening Doors scholarship was available to members of the program group only. 
Both program and control group members were potentially eligible for federal Pell Grants and 
other financial aid, but those programs were administered separately from the Opening Doors 
scholarship. (As noted elsewhere in this report, the Opening Doors scholarship was designed as 
a supplement to existing financial aid, not as a replacement.) In practical terms, this meant that 
the Opening Doors scholarship should increase the amount of money available to students in the 
program group. An important component of the program was that the money from the Opening 
Doors scholarship was paid directly to the student and could be used for school expenses along 
with basic living expenses and other needs. This was also very different from Pell Grants, which 
went directly to the institution to cover tuition and fees, and may not have felt as tangible to stu- 
dents. (Chapter 4 provides more information about the program’s effect on enrollment.) 

Table 3.3 presents information from Delgado Community College on the impact of the 
Opening Doors program on financial aid other than the Opening Doors scholarship. (Compara- 
ble data were not available for LTC.) The first column shows the percentage of program group 
members who were awarded financial aid, including Pell Grants, loans, and work-study funds. 
The second column presents the same information for control group members, and the third 
column shows the difference between the two. Where the differences are statistically signifi- 
cant, they are marked by asterisks. (See Box 3.1 on page 41 for an explanation of how to in- 
terpret an impact table.) 

Table 3.3 also shows that in the first semester after students were randomly assigned, a 
slightly higher percentage of program group members registered for classes than did control 
group members (85 versus 80 percent), and were more likely to register for classes full time. 
Despite these differences, students in the program and control groups were awarded financial 
aid at virtually identical rates. This was an expected result. Students typically apply for financial 
aid several weeks (if not months) before the start of the semester, and the staff from the Open- 
ing Doors program did not have any influence on students’ decisions to apply or whether they 
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The Opening Doors Demonstration 
Table 3,3 

Impacts on Non-Opening Doors Financial Aid During the First Two Semesters 
Following Random Assignment at Delgado Community College 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


First semester 










Registered for any class (%) 


84.5 


80.1 


4.4 * 


2.4 


Enrolled full time 


60.6 


55.0 


5.7 * 


3.4 


Enrolled part time 


23.8 


25.1 


-1.3 


2.9 


Awarded financial assistance (%) 


71.9 


69.4 


2.5 


2.9 


Awarded Pell Grant 


70.4 


67.2 


3.2 


3.0 


Awarded any other grants’* 


7.7 


5.1 


2.6 


1.7 


Awarded subsidized loans 


43.6 


47.2 


-3.7 


3.3 


Awarded unsubsidized loans 


17.5 


18.3 


-0.8 


2.6 


Awarded Federal Work-Study 


4.5 


3.4 


1.1 


1.3 


Total financial aid award** ($) 


2,112 


2,083 


29 


114.1 


Grants/scholarships (%) 


64.1 


59.7 






Loans (%) 


33.5 


38.3 






Federal Work-Study (%) 


2.4 


2.0 






Second semester 










Registered for any class (%) 


69.0 


55.9 


13.0 *** 


3.2 


Enrolled full time 


48.0 


33.8 


2 *** 


3.4 


Enrolled part time 


21.0 


22.1 


-1.1 


2.8 


Awarded financial assistance (%) 


63.8 


51.6 


2 *** 


3.3 


Awarded Pell Grant 


61.8 


49.4 


12 5 *** 


3.3 


Awarded any other grants’* 


10.4 


7.2 


3.2 


1.9 


Awarded subsidized loans 


46.6 


38.7 




3.3 


Awarded unsubsidized loans 


18.7 


18.2 


0.5 


2.7 


Awarded Federal Work-Study 


6.9 


5.6 


1.3 


1.7 


Total financial aid award** ($) 


2,011 


1,699 


313 ** 


125.4 


Grants/scholarships (%) 


53.9 


52.2 






Loans (%) 


42.0 


43.6 






Federal Work-Study (%) 


4.1 


4.3 






Sample size (n = 817) 


406 


411 







(continued) 



39 








Table 3.3 (continued) 



SOURCES: MDRC calculations from financial aid and transcript data from Delgado Community College. 

NOTES: Distributions may not add to 100 percent because of rounding. 

A two-tailed t-test was applied to differences between the research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“Other grants include the Leveraging Education Assistance Partnership (LEAP) grant and Supplemental 
Equal Opportunity Grant. 

'’These financial aid amounts do not include the Opening Doors scholarship. Characteristics shown in italic 
type are calculated for a proportion of the full sample. These estimates are nonexperimental and do not have a 
causal interpretation. Therefore, these cells are left blank. 



were awarded financial aid packages. (The Opening Doors scholarship was not counted as regu- 
lar financial aid, and is not shown in the table.) 

The bottom panel of Table 3.3 shows the impact of Opening Doors on registration and 
financial aid in the second semester after students were randomly assigned. By that point, there 
was a widening gap between the program and control group members in college registration 
rates: 69 percent versus 56 percent, respectively. The table also shows that students in the pro- 
gram group were more likely to be awarded financial aid during the second semester than stu- 
dents in the control group: 64 percent versus 52 percent, respectively. This result is primarily 
due to the program’s effect on retention. In other words, more program group members than 
control group members made the decision to re-enroll in college during their second semester, 
and thus were also more likely to be awarded financial aid. 

Differences in Receipt of Counseling and Advising Services 

Similar to the Opening Doors scholarship, the Opening Doors counseling services were 
available only to students in the program group. Unlike the scholarship, however, counseling 
was generally available to all students in other places on campus, though college administrators 
described such services as weak. LTC relied on individual faculty members to provide academ- 
ic advising to students, and employed no other counselors on staff At Delgado’s City Park 
campus, 12 academic advisers were employed to serve a student population of more than 
11,000. Students could make appointments with these advisers, but generally did not have a 
continuing relationship with them. Delgado conducted an Orientation, Advising, Registration, 
and Support Services (OARS) program for all new students, which consisted of group and in- 
dividual counseling, help with course registration, and a campus tour. After declaring a major. 
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Box 3.1 



How to Read the Tables in This Report 

Most tables in this report use a similar format, illustrated below. The abbreviated table shows 
some educational outcomes for the program group and the control group. The first row of data, 
for example, shows that about 85 percent of the program group members and about 80 percent 
of the control group members registered for any courses in the Louisiana colleges during their 
first semester in college. 

Because individuals were assigned randomly either to the program group or to the control 
group, the effects of the Opening Doors scholarship and counseling can be estimated by the dif- 
ference in outcomes between the two groups. The “Difference” column in the table shows the 
differences between the two research groups’ outcomes — that is, the program’s estimated im- 
pacts on the outcomes. For example, the estimated impact on registering for any classes in the 
Louisiana colleges can be calculated by subtracting 80.1 percent from 84.5 percent, yielding an 
estimated increase (or “impact”) of about 4 percentage points. 

Differences marked with asterisks are “statistically significant,” meaning that there is only a 
small probability that the difference occurred by chance rather than as a result of the program. 
The number of asterisks indicates the likelihood of this chance (that is, the likelihood that there 
is actually no difference between the program and control group outcomes). One asterisk cor- 
responds to the 10 percent level, or a 10 percent chance that there is no difference between the 
program and control group outcomes; two asterisks, a 5 percent chance; and three asterisks, a 1 
percent chance. For example, as the second row of data shows below, the Opening Doors pro- 
gram increased the likelihood that a student enrolled full time by 5.7 percentage points, a differ- 
ence that is statistically significant at the 10 percent level, indicating that there is a 10 percent 
probability that the difference occurred by chance rather than as a result of the program. 

The statistical significance is calculated using the standard error of the impact estimate, shown in 
the rightmost column. The standard error is a measure of uncertainty or variability around the 
impact estimate. Two useful rules of thumb is that there is a good (about 90 percent) chance that 
the hue impact is within plus or minus 1.65 standard errors of the estimated impact at the 10 per- 
cent significance level and a very good (about 95 percent) chance that the true impact is within 
plus or minus 2 standard errors of the estimated impact at the 5 percent significance level. For 
example, in the second row of data below, there is roughly a 90 percent chance that the impact of 
the program on registration lies between 0.09 and 1 1.3 percentage points, calculated as the im- 
pact (or difference) plus or minus (1.65 multiplied by the standard error), or 5.7 ± (1.65 x 3.4). 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


First semester 










Registered for any class (%) 


84.5 


80.1 


4.4 * 


2.4 


Enrolled full time 


60.6 


55.0 


5.7 * 


3.4 


Enrolled part time 


23.8 


25.1 


-1.3 


2.9 
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students were expected to get most of their information about which courses to take from facul- 
ty advisers, similar to what was done at LTC. In the words of a senior Delgado administrator: 

Advising and mentoring is the college’s weakest area. Most students are on 
their own, choose classes purely [based] on schedule or friends... Most ad- 
vising is given by instructors, but is infrequent and unreliable for most non- 
health programs. There’s very little tme advising on campus. 

Did the Opening Doors counselors add value to the colleges’ existing services? The 
most reliable answer comes from students’ responses to the Opening Doors follow-up survey, 
shown in Table 3.4. When asked if they had a college contact for advice and support on person- 
al matters, program group members were more likely than control group members to respond 
affirmatively. Program group members were also more likely to report that they had a mentor at 
the college for academic and career-related matters. Finally, program group members reported 
that they received more help with financial aid counseling, academic advising, tutoring, career 
counseling, and job placement assistance than control group members. Some of this support 
may have been provided by Opening Doors counselors, though it probably also reflects the fact 
that program group members were more likely than control group members to re-enroll in col- 
lege after one semester, giving them more opportunities to interact with faculty advisers and 
other student services staff 



Student Perceptions and Experiences 

As noted in Chapter 2, intensive qualitative interviews were conducted with 50 women 
who were randomly selected from the Opening Doors sample. The interviews took place after 
Hurricane Katrina, and were mainly focused on the women’s adaptation and resilience after the 
storm. Not all the women in the program group were asked about their involvement with Open- 
ing Doors, but in several interviews where it came up, the reactions were favorable.^ 

A couple of the women in the qualitative study briefly mentioned getting the Opening 
Doors scholarship and using it to make purchases for their children or other family members. A 
few others spoke at greater length about both the money they received and the assistance pro- 
vided by program staff. For example, a 28-year-old respondent spoke about how Opening Doors 
helped her at a time when her life was in upheaval. She had just left her husband after discover- 
ing that he was cheating on her, and decided she wanted to go back to school. In her words: 



^In a few instances, problems with the audiotape made it difficult to discern what some women in the pro- 
gram group had to say about Opening Doors. 
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The Opening Doors Demonstration 
Table 3.4 

Students' Responses to Follow-Up Survey: Impacts on Counseling and Advising 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Sample 

Size 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


12-month impacts on counselins 

College contact for advice and 


support for personal matters (%) 


789 


53.0 


44.3 


g y ** 


3.5 


Mentor at college available for academic 


and career-related matters (%) 


793 


50.9 


43.1 


y g ** 


3.5 


Number of times service was received” 


Financial aid counseling 


796 


2.4 


1.8 


Q 5 


0.1 


Academic advising 


796 


2.2 


2.0 


0.3 * 


0.1 


One-on-one tutoring 


796 


2.1 


1.8 


0.3 * 


0.2 


Career counseling 


796 


1.7 


1.4 


0.3 ** 


0.1 


Job placement assistance 


796 


1.2 


0.9 


0.2 ** 


0.1 


Information on transferring 


796 


1.0 


0.8 


0.1 


0.1 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“The total number of counseling visits cannot be obtainined by summing across categories because different 
areas of counseling do not necessarily represent unique visits. 



I was looking at the new things Delgado had. And then I met this girl in the 
[registration] line and she was like, they have this program called Opening 
Doors and that’s how it happened. And actually it was a great help. ’Cause I 
wasn’t with him [my husband] no more and you know how men can be 
mean. He wasn’t giving me any money for the bills. And so that money came 
in handy. . . . 

This respondent went on to say that Opening Doors staff told her that if she “needed 
anything to come and ask them.’’ She also said she got e-mails from her counselor “all the time” 
to let her know about important events and deadlines on campus. 

Another student, age 27, mentioned the monitoring role that Opening Doors staff at 
Delgado played. When asked what kind of help she got from the program, she explained it this 
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way: “They checked up on me a lot to make sure I kept my grades up. For keeping my grades 
up and coming to school as often [as possible], try to be there every day, they gave us a small 
check every semester. For like $250.” 

The 27-year-old went on to say that as “nice” as it was to get the cash, she also appre- 
ciated the encouragement she received from the program staff: “You always need more en- 
couragement when you want to do something.” 

The most extensive comments about Opening Doors came from a 31 -year-old student. 
She began by telling the interviewer how she used the scholarship: 

Student: They assisted me with a scholarship when I was at Delgado, which 
really helped me financially to get through some hard times. I was able to 
get my books, some clothes so we could [get] back and forth to school. And 
I think the last payment I got — they give you so much at the end of the 
semester. Well, that semester was around Christmas, so it helped me to get 
my children some Christmas things. 

Interviewer: Do you think it made a difference? 

Student: It did. It was an incentive to go. And you know you have to make 
a certain grade point average, so it would help you keep your grades up. 

And they offered assistance not just financially. If you needed to talk, the 
counselors were always there to talk. If you needed help like with tutoring 
in your classes, they had assistance in any area. So it wasn’t just, “Okay. We 
want to give you this grant and that’s it.” They looked at your whole situa- 
tion: your kids, if you needed child care, if you needed help getting back and 
forth to school, whatever. And if you have kids and you’re trying to go to 
school, there’s barriers that will keep you from going or have you drop out, 
because you don’t have that help. So that program did everything that would 
be able to assist me to stay in school. 

This woman’s experience suggests a more personalized and involved relationship with 
her counselor than MDRC’s observations suggested took place for most students. Unfortunate- 
ly, most of the interviews did not probe on this topic, so it is impossible to know whether her 
experience was unusual or was shared by other program group members. 

Motivations for and Chaiienges to Earning a Coiiege Degree 

Many of the interviews delved into the reasons why women decided to go to college. 
The most common reaction from both program and control group members was the desire to 
provide a better life for their children. In addition, some women mentioned having specific ca- 
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reer goals or wanting to escape menial jobs. Financial independence from men or from public 
assistance programs provided motivation for a few women. For example, a 30-year-old control 
group member explained her decision to go to school this way: 

Fm the kind of person that I don’t like to ask anybody for anything, even my 
husband. Fm just like that. Fve been independent. Fve been working all my 
life. So, when I want something for my kids, which I think they would need 
later on down in life or it’s essential for them, I want to do it for them. When 
I can’t do it, it frustrates me. So, that’s one thing, I wanted to go to school. I 
want to be able to reach for cash when I want for my children. 

It was clear from the interviews that many of the women faced serious obstacles to 
earning a college degree, even before they were set back by Hurricane Katrina. The challenge of 
juggling work, child care, and other family responsibilities while going to school was a constant 
theme. Some of the women had supportive husbands or partners, but many others were single or 
appeared to be in unsupportive relationships. Most striking was the frequency with which inter- 
viewees described significant health problems afflicting themselves or other close family mem- 
bers. Asthma and diabetes were mentioned most often, and serious kidney or heart problems 
were mentioned a few times. A handful of women had children with significant needs: a deaf 
son, a son with a rare disease that left his legs paralyzed, a son with sickle-cell anemia. Finally, 
a few of the women described serious emotional scars left from physical or sexual abuse from 
their own childhoods. 

For most of the women in the qualitative sample, the path toward earning a college de- 
gree involved more than one institution. Women who attended only Delgado or Louisiana 
Technical College-West Jefferson were a distinct minority. Displacement caused by Hurricane 
Katrina contributed to this trend, but the pattern was evident even before the storm. The reasons 
for switching colleges were as varied as the women themselves. In some cases, women indi- 
cated that they attended different schools when they selected a new career path or major. In oth- 
er cases, their decisions appeared to be driven by financial concerns, such as not being able to 
afford tuition at more expensive four-year institutions. Some women stopped attending particu- 
lar colleges because they were not happy with the teachers or the quality of education they re- 
ceived, and a few indicated that they left because of failing grades. 

Disruption to Educational Plans Caused by Hurricane Katrina 

Virtually all the women who were attending college at the time of Hurricane Katrina 
had their educational plans derailed: sometimes by a semester, sometimes by a year or more. 
Even if the women escaped the hurricane without suffering significant personal losses, the col- 
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leges they attended were not able to restore classes and services for several months. A 24-year- 
old control group member attending Delgado described her experience this way: 

School was going good. I had all the classes that I wanted, then when Hurri- 
cane Katrina hit, everything was wiped out and I had to start all over again. 

The school screwed up my schedule. They messed up everything. They were 
so unorganized. When I went back to register, they placed me in two math 
classes at the same time, which wasn’t my fault and caused me to fail one 
math class. I didn’t even know I was registered in that class. So I have to go 
and appeal that and see about getting something done. I spoke to someone 
over [in] the math department and by coincidence it was her class that I was 
in. So I have to go do something about that because I have a big withdrawal 
on my records for that class. I withdrew right before she gave me an “F.” 

A 24-year-old program group member talked about the challenges she faced getting 
herself and her son back in school: 

That spring Delgado was opening back up. They were holding classes only in 
Building 1, ’cause all the rest of the buildings were still under constmction. 

And still now most of the buildings are still messed up. Like the library, you 
can’t go in the library at all. In spring they were opening back up. So we 
ended up going to school but I didn’t go full time because I still had my son 
and my mom was watching him for me and I was trying to get him registered 
with the school board, which is a lot of running around. ’Cause he had to take 
a test and they had [to] evaluate him to see if they wanted to put him in spe- 
cial education classes or not. 

Like the two women quoted above, many respondents in the qualitative sample seemed 
determined to re-enroll in college as soon as they could after the hurricane. In addition to Del- 
gado, some of the more common choices included the University of New Orleans (part of the 
Louisiana State University system) and the Southern University at New Orleans (a historically 
black institution). Other women moved away and enrolled in community colleges or universi- 
ties in Texas, California, or Michigan. In some cases, women expressed a desire to return to 
New Orleans, though a number of others indicated they were happier in their new homes, usual- 
ly because they perceived the educational and employment opportunities to be better. 

Not all the women were able to go back to school or make new plans after the hurri- 
cane. For some, the personal losses they experienced or the demands of caring for other family 
members were too overwhelming. Feelings of stress and depression came up repeatedly in the 
interviews. For example, when asked about the hardest things after the hurricane, a 27-year-old 
program group member responded: 
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I thought it was not going to get better. It was depressing. In between De- 
cember until the middle of last year, I stayed depressed for a long time. I’m 
very good at holding things together but I was depressed. I didn’t go back to 
school last year [2006] because of trying to get everything back together. I 
knew I couldn’t handle it. I have to work on one thing at a time. 

Another program group member, age 21, was asked a similar question about whether 
her life had gotten better or worse since the hurricane. (This respondent had moved to Dallas.) 
She replied: 



Um, I would say my financial situation could possibly be better, but then at 
the same time I had more stability back before the hurricane, you know. I 
was on the track that I needed to be on. I was working. I had money. My kids 
were in day care. I had assistance for that. Here it’s totally opposite. I’m 
working. I have the money, but I’m shelling out, you know, the money to 
make sure that my life stays together, and I’m still not meeting my educa- 
tional goals, you know. 

A handful of women described such severe physical and emotional problems after the 
hurricane that it was questionable whether they would ever fully recover. It was encouraging, 
however, that a majority of the women expressed determination to move on with their lives, 
despite the trauma they had experienced. A 29-year-old program group member reflected this 
resilience when she responded to a question about her life in the aftermath of the hurricane: 

You just have to pick yourself back up. You just have to pick up — it’s like a 
broken puzzle and you just have to pick it up, pick up and just put it back to- 
gether. And that’s how it feels to me, like you’re just putting back the pieces 
of your life. . .You want to pick up and get to where you was before because 
of your children and your family and stuff, too. . . And that’s all I’m trying to 
do is just start back over. Start over and get back to where I was before. 

The above respondent was not enrolled in college at the time of the interview, but 
planned to go back and earn a bachelor’s degree at a New Orleans university. 

The desire to earn a college degree was particularly noteworthy given the fact that 
many of the women faced serious obstacles to attending college. The challenges of juggling 
school, work, child care, and other family responsibilities — both before Hurricane Katrina and 
after — came up repeatedly in the interviews. 

Again, the qualitative interviews were conducted with just 50 women. The findings 
cannot be used to generalize across the full Opening Doors sample, which also included some 
men. Nonetheless, the interviews provide a more complete portrait of the women’s lives, both 
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on- and off-campus. For those in the program group, Opening Doors appears to have offered 
meaningful help. At the same time, Opening Doors represented only a small part of the wom- 
en’s college experience. Other factors — including Hurricane Katrina — may have had far 
greater influence, for better or worse, on students’ ability to accomplish their academic and per- 
sonal goals. The next two chapters examine the effects of the program on educational, social, 
psychological, and health outcomes for all students in the study. 



48 




Chapter 4 

Academic Impacts 



While several studies of the relationship between financial aid and access to college 
have been undertaken, few studies have been conducted on the relationship between finan- 
cial aid and academic success.' Questions about persistence and retention in relation to fi- 
nancial aid receipt have been difficult to answer because of a lack of longitudinal data and 
problems with disentangling the factors that are associated with financial need from those 
associated with academic success. That is, many of the factors linked to financial need are 
also related to lack of academic success. This chapter analyzes the effect of the Opening 
Doors incentive scholarship and counseling program on educational outcomes as far as sev- 
en semesters after students enrolled in the study. It attempts to parse out the influence of 
Hurricane Katrina in several ways to show the effect of the program on effort — and subse- 
quent success — in an academic setting. 

The key findings are as follows: 

• An analysis of the first two semesters of the Opening Doors intervention (the 
pre-Katrina period) suggests that the incentive scholarship and counseling in- 
tervention encouraged program group students to register at greater rates dur- 
ing both the first and second semesters, while the program was in place. The 
findings suggest that program group students were more likely to register full 
time, cam more credits, and meet the 2.0 grade point average (GPA) bench- 
mark set by the program. 

• The enrollment data suggest that the intervention increased persistence 
among program group students, as the largest effect occurs in the second 
semester after random assignment. The evidence also suggests that the im- 
pact may be larger when the second program semester takes place during the 
summer, which is considered by some to be a nontraditional semester and 
one that students often bypass. Significant but more moderate effects on 
second-semester enrollment are also seen when the second semester is in the 
fall or spring. 

• Transcript data also suggest positive effects on academic outcomes as late as 
the fourth semester after random assignment — two full semesters after most 
students had exhausted their program eligibility. 



'Bellinger (2004). 
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• Overall, the program resulted in an increase of about four credits (or one- 
third of a semester) for program group students over seven semesters. While 
the impacts after Hurricane Katrina are smaller than those before the hurri- 
cane occurred, it is not clear that the decline can be completely attributed to 
the disaster, as there was evidence of a decrease in the impact on enrollment 
before the hurricane struck. 

The remainder of Chapter 4 proceeds as follows: First, the effect of Hurricane Katrina 
on the analyzed data is discussed, along with an introduction to this chapter’s key questions. 
The findings for the pre- and post-Katrina periods for each of the key questions are then pre- 
sented. Next, the lessons learned from the data are summarized, and the chapter concludes with 
a discussion of possible reasons for the findings. 



Data and Outcomes 

Assessing the Effect of Hurricane Katrina 

Transcript data from the schools in which the study was implemented. National Student 
Clearinghouse data, and results from a student survey are analyzed in this chapter. Transcript 
data are available from the point of random assignment through the seventh semester after ran- 
dom assignment for all 1,019 sample members. As discussed in earlier chapters, this period in- 
cludes Hurricane Katrina, which stmck the Gulf of Mexico in August 2005. Given the delete- 
rious effect of Hurricane Katrina on general registration at both Louisiana schools (see Figure 
1 .3 in Chapter 1 for the effect of the hurricane on enrollment at Delgado Community College), 
it is clear that all student outcomes from the post-Katrina period may be severely affected by the 
storm. In addition, the schools largely suspended submitting data to the Clearinghouse imme- 
diately following the storm. As a result, the quality of the Clearinghouse data may have de- 
clined in the immediate aftermath of the storm, possibly causing estimates based on those data 
to differ slightly from estimates based on transcript data. 

Parsing the effect of Katrina is therefore important to unmasking the effect of the Open- 
ing Doors program in Louisiana. This is done in several ways in this chapter. First, the pre- 
Katrina period for all cohorts is examined. Figure 4.1 illustrates this approach. The figure shows 
that at least two semesters of administrative data were collected for each of the four study co- 
horts prior to August 2005. Those two semesters are used to provide a look at the program 
while it was in operation, using the full sample of 1,019 students. This analysis provides a clean 
look at impacts prior to the hurricane. 

Second, the pre-Katrina period for just the first and second cohorts is analyzed (a partial 
sample of 537 students), as illustrated in the left half of Figure 4.2. These cohorts permit the 
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The Opening Doors Demonstration 
Figure 4,1 

Illustration of the Research Sample for the Pre-Katrina Analysis 
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longest period of time to analyze the effects of the intervention after the scholarship and coun- 
seling ended, which enables an assessment of whether effects attenuate over time. These results 
are also more directly comparable to those in the early findings research brief, as the same sam- 
ple is followed over a longer period of time.^ 

Next, the post-Katrina period is analyzed using the first and second cohorts and data cov- 
ering the period after August 2005, as shown in the right half of Figure 4.2. Finally, cumulative 
outcomes are presented showing the total effect of the intervention over the follow-up period. 

Data Sources 

While transcript data reflect academic achievement, it is important to note that the data 
show only the progress and achievement at a particular institution. Students, especially commu- 
nity college students, are very mobile and enroll in several institutions, sometimes simulta- 
neously.^ Some evidence of this “churning” among a random sample of 50 women who partici- 
pated in intensive qualitative interviews was presented in the previous chapter. As a result, data 
from the Clearinghouse, which allow the tracking of school enrollment and graduation from 
multiple institutions over time, are also employed. Note, however, that those data are only ana- 
lyzed for participants who started at Delgado Community College — the larger institution in the 
study, providing 80 percent of the sample members."^ 

The survey, intended to illuminate the effects of the Opening Doors programs in gener- 
al, is particularly affected by Hurricane Katrina, as detailed in Chapter 2. Survey data provide 
information on other outcomes related to academic success such as the level of effort. Chapter 2 
describes in detail the sample used in this chapter. 

This chapter focuses on the following questions: 

• Did the intervention increase academic achievement at the study institutions? 

• Did the intervention encourage students to persist? 

• Were levels of effort (for example, hours studying, working fewer hours) af- 
fected by the program? 



^Brock and Richburg-Hayes (2006). The research brief presented findings for the first three semesters for 
the spring 2004 and summer 2004 cohorts only (representing 537 observations). The study found that program 
group students in these cohorts were more likely to enroll in college Ml time, pass more courses and earn more 
course credits, and have higher rates of registration in college in the second and third semesters after random 
assignment compared with control group students. 

^Peter and Cataldi (2005). 

"^Recall that Louisiana Technical College (LTC) does not report to the National Student Clearinghouse. 
While there are Clearinghouse records for those students who attended LTC if they enrolled in another institu- 
tion that does report, those records are not utilized in the analyses in this chapter. 
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Early Academic Outcomes Prior to Hurricane Katrina 



Impact on Academic Achievement at the Study Institutions 

Table 4.1 shows educational outcomes for students randomly assigned to be eligible for 
the Opening Doors program and those randomly assigned to receive whatever financial aid and 
counseling were normally available to students. The outcomes in this table and Table 4.2 are 
limited to the first two semesters, reflecting the period in which program students were eligible 
to receive the scholarship and counseling. The top panel of the table shows that large propor- 
tions of both the program group and the control group registered during the first semester of 
random assignment. Note that “registration” is defined as students being enrolled after the 
“add/drop” period — that is, a brief period of time following enrollment during which students 
could elect to add or drop a course for which they had registered — which typically occurred 
five days after the start of the semester. While the point of random assignment generally oc- 
curred after students had already decided to matriculate, about 18 percent of students in the pro- 
gram declined to register compared with slightly more than 23 percent of control group stu- 
dents. This pattern suggests, first, that the eligibility for a contingent source of funds can have a 
modest effect on registration even when students may have independently decided to attend col- 
lege. Registration among program group members is 5.3 percentage points higher than that of 
control group members. How important is a 5.3 percentage point difference? In the absence of 
the program, more than 23 percent of students did not register for the first semester despite be- 
ing motivated to attend; therefore, the availability of the Opening Doors intervention reduced 
this gap to about 18 percent (or close to one-fourth reduction). 

The remaining rows of the top panel of Table 4.1 show that program students were 
more likely to register full time (that is, register for 12 or more credits) and attempt more regular 
credits (or college-level classes that would count toward a degree). Program group students 
earned more of both college-level credits and developmental credits (which do not count toward 
a degree). The panel shows that program students were more likely, by nearly 1 1 percentage 
points, than control group students to attain at least the benchmark GPA (2.0) of an average 
grade of “C” or better, suggesting that the Opening Doors program increased the number of stu- 
dents meeting this benchmark by 24 percent. 

The second panel of Table 4.1 shows that the program group was more likely to regis- 
ter the next semester, a common measure of persistence.^ Program group students were 15 
percentage points more likely to register for a course and more likely to enroll full time during 



^Note that summer is counted as a semester similar to the fall and spring terms. 
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The Opening Doors Demonstration 
Table 4.1 

Educational Outcomes, First and Second Semesters: All Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


First semester 


Registered for any courses (%) 


82.2 


76.8 


5.3 ** 


2.3 


Enrolled full time”' (%) 


60.2 


53.9 


6.4 ** 


3.0 


Enrolled part time'’ (%) 


22.0 


23.0 


-1.0 


2.5 


Number of credits attempted 


8.6 


8.0 


0.6 ** 


0.3 


Regular credits 


5.6 


5.1 


0.5 * 


0.3 


Developmental credits 


2.9 


2.9 


0.1 


0.2 


Number of credits earned 


5.9 


4.6 


j 2 *** 


0.3 


Regular credits 


4.1 


3.1 


0 g 


0.2 


Developmental credits 


1.8 


1.5 


0.3 ** 


0.1 


Withdrew from one or more courses (%) 


27.0 


30.8 


-3.8 


2.8 


Term GPA (%) 


No GPA*’ 


23.6 


31.3 




2.6 


Earned a 2.0 GPA or greater 


54.8 


44.2 


10.6 *** 


2.9 


Earned less than a 2.0 GPA 


21.6 


24.5 


-2.9 


2.5 


Second semester 


Registered for any courses (%) 


64.8 


49.8 


15 0 *** 


2.8 


Enrolled full time”' (%) 


46.8 


31.5 




3.0 


Enrolled part time'’ (%) 


18.0 


18.3 


-0.4 


2.3 


Number of credits attempted 


6.2 


5.0 


1 2 *** 


0.3 


Regular credits 


4.5 


3.6 


0 ^ 


0.3 


Developmental credits 


1.7 


1.4 


0 3 


0.1 


Number of credits earned 


3.9 


2.8 


1 1 *** 


0.3 


Regular credits 


3.0 


2.1 


0 ^ *** 


0.2 


Developmental credits 


0.9 


0.7 


0 3 *** 


0.1 


Withdrew from one or more courses (%) 


25.8 


23.7 


2.1 


2.5 


Term GPA (%) 


No GPA" 


42.0 


56.4 


_14 4 *** 


3.0 


Earned a 2.0 GPA or greater 


38.0 


26.9 


^ ^ 0 *** 


2.8 


Earned less than a 2.0 GPA 


20.0 


16.7 


3.4 


2.3 


Sample size (n = 1,019) 


505 


514 







(continued) 
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Table 4.1 (continued) 

SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College transcript 
data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

^To be considered full-time, students were enrolled in 7 or more credits. 

’’To be considered part-time, students were enrolled in 6 or fewer credits. 

‘’The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 



the second semester — a semester in which they were still entitled to the scholarship. Program 
group students continued to register for and earn more credits and, again, were more likely to 
earn a semester grade average of “C” or higher. 

Impact on the Acquisition of Postsecondary Education More Generally 

Data from the Clearinghouse, which show enrollment at other institutions in addition to 
the intervention institutions, are presented in Table 4.2 to provide a more comprehensive 
measure of attainment than those based on transcript data. As mentioned earlier. Clearinghouse 
data are analyzed only for Delgado. The first outcome, “At any school,” represents data from 
both the Clearinghouse and transcripts. The second outcome, “At Delgado Community 
College,” represents enrollment based on Clearinghouse data only, while the outcome, “For any 
course at Delgado,” relies on transcript data only. This last variable is presented for comparison 
purposes. The top panel of the table shows that students largely registered only at Delgado 
during the first semester (over 80 percent of students across all data sources). The transcript 
registration variable is similar in magnitude to those from the Clearinghouse, but there are small 
differences between the data sources in the estimates for the control group. 

The second panel of the table shows a considerable drop in registration from the first to 
the second semester for both the program and control group students. However, the impact of 
the intervention is large: Delgado program group students were about 13 percentage points 
more likely to register than control group students this semester. Again, a small difference be- 
tween the Clearinghouse and transcript data sources is evident, although both suggest large, sta- 
tistically significant impacts. Overall, the table shows that the Opening Doors program in Loui- 
siana increased enrollment, particularly in the second semester, even when considering enroll- 
ment using a measure that reflects a broader set of institutions. 

In summary, a look at the first two semesters of the Opening Doors program (the pre- 
Katrina period) suggests that the program encouraged students to register at greater rates during 
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The Opening Doors Demonstration 
Table 4.2 



Enrollment at Delgado Community College and Other Institutions 
for the First Two Semesters: All Cohorts 



Delgado Community College and Louisiana Technical College 



Outcome (%) 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


First semester 

Registered 


At any school 


84.5 


81.0 


3.4 


2.4 


At Delgado Community College 


84.5 


81.0 


3.4 


2.4 


For any course at Delgado (transcript) 


84.5 


80.1 


4.4 * 


2.4 


At any 2-year school 


84.5 


81.0 


3.4 


2.4 


At any 4-year school 


0.2 


0.2 


0.0 


0.3 


Second semester 

Registered 


At any school 


69.5 


56.7 


12.8 *** 


3.2 


At Delgado Community College 


69.0 


56.2 


12.8 *** 


3.2 


For any course at Delgado (transcript) 


69.0 


55.9 


13.0 *** 


3.2 


At any 2-year school 


69.2 


56.4 


12.8 *** 


3.2 


At any 4-year school 


0.5 


0.2 


0.3 


0.4 


Sample size (n = 817) 


406 


411 







SOURCES: MDRC calculations using data from the StudentTracker service of the National Student 
Clearinghouse and transcript data from Delgado Community College. 

NOTES: Calculations for this table used all available data for the 8 1 7 sample members who were randomly 
assigned at Delgado Community College. 

The Clearinghouse collects data from about 3,100 colleges that enroll 91 percent ofU.S. college students 
(http://www.studentclearinghouse.org/about/pdfs/Clearinghouse_profile.pdf). Students have the right to opt 
out of having their information sent to the Clearinghouse. MDRC was able to find records in the 
Clearinghouse file for 93 percent of the students randomly assigned at Delgado Community College. 

A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; and * = 10 percent. 

Distributions may not add up to 100 percent because of rounding. 
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both the first and second semesters, when the program was in place. The findings suggest that 
program group students were more likely to register full time, earn more credits, and meet the 
GPA benchmark set by the program. 

What May Be Driving These Results? 

The findings presented thus far in this chapter and in Chapter 3 suggest that the Open- 
ing Doors program altered college-going behavior. However, it appears that these effects are 
more pronounced for some cohorts than for others, possibly reflecting differences in academic 
terms. Figure 4.3 shows registration patterns over time for each of the four cohorts in the study. 
The first panel shows that for the spring 2004 cohort there is a large, statistically significant dif- 
ference (close to 24 percentage points) in enrollment between the program group and control 
group during the second semester or summer term. A look at the next two panels shows a 
smaller difference in second semester enrollment among the summer 2004 cohort (about 1 1 
percentage points) and a modest difference in second semester enrollment among the fall 2004 
cohort (about 8 percentage points) — both of which are statistically significant. The last panel, 
for the spring 2005 cohort, again shows a large impact on second semester enrollment (slightly 
more than 25 percentage points). Appendix Table C.l shows the registration numbers upon 
which these figures are based. 

While these results suggest that paying for persistence may go further if pay is granted 
for summer attendance, such a conclusion is not definitive. First, the effect of summer cannot be 
cleanly disentangled from the effect of the payment. That is, larger impacts in summer are main- 
ly evident when summer is the second semester as opposed to when summer is either the first or 
third semester. Second, the last cohort (the spring 2005 cohort) was required to enroll during the 
summer term in order to be eligible for the Opening Doors program as a result of the scheduled 
conclusion of the program, making it difficult to know what might have happened had they also 
been eligible for the scholarship in the fall. Finally, Appendix Table C.l suggests that the sum- 
mer 2004 cohort has impacts that persist throughout all the semesters before Hurricane Katrina. 
(Figure 4.3 shows that Katrina occurred prior to the fifth semester of this cohort.) Analyses (not 
shown) suggest that this cohort also differs from the other cohorts on a few observable baseline 
characteristics, such as age (students in the summer cohort were younger) and prior college expe- 
rience (students in the summer cohort were more likely to have reported that they’d earned some 
college credits prior to random assignment). These differences may indicate that students in this 
second cohort were more motivated than students recmited in other semesters, which further 
complicates interpretation of the bigger impacts observed during the summer term.® 



®It is important to keep in mind that, while random assignment ensures that program group and eontrol 
group members within a eohort should be identieal (on average), there may be differenees across eohorts. 
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Percentage Registered, by Cohort 
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= 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 



In summary, the enrollment data suggest that the intervention’s largest effect may lie in 
encouraging program group students to take courses during the summer term while the program 
is in operation. While there are still significant effects on enrollment when the spring cohorts are 
omitted, the impacts are much more moderate. 



Later Academic Outcomes Prior to Hurricane Katrina 

The analysis above concentrated on just the first two semesters while the program was in 
operation. A logical question is: What happens once the scholarship and counseling end? An- 
swers to this question are provided by analyzing the first two cohorts of the study. As was shown 
in the left panel of Figure 4.2, the first two cohorts contain the most semesters of pre-Katrina da- 
ta, allowing a look at what happens once the program’s components are largely removed.’ 

Academic Achievement and Persistence at the Study Institutions 

Table 4.3 shows academic outcomes for the first four semesters after random assign- 
ment for just the first two (spring 2004 and summer 2004) cohorts. This table is similar to Table 
4.1, but uses about half of the observations (N = 537) and shows outcomes for the third and 
fourth semesters after the intervention.* The top panel of the table shows no significant differ- 
ence in overall registration rates between program group students and control group students. 
However, program group students were more likely to register full time, by about 9 percentage 
points. Program group students attempted more college-level credits and earned more of both 
college-level and non-degree-applicable (developmental) credits. Unlike the evidence using the 
full sample, in the first two cohorts program group students were less likely to withdraw (by 
about 7 percentage points) than control group students from one or more courses during the first 
semester. This finding could be interpreted positively or negatively. A positive interpretation 
suggests that students are not overwhelmed and can effectively handle their attempted course 
load, requiring less subsequent adjustment through course withdrawals. A negative interpreta- 
tion could be that students simply registered for easier courses to begin with, resulting in less 
need to reassess their registration choices. The data cannot distinguish between these alternative 
interpretations. 



’while the intervention lasted only two semesters, Appendix Table C.2 shows that about 10 percent of 
students (26 students) in the first two cohorts received scholarships in the third semester and slightly more than 
1 percent (or 3 students) received scholarships in the fourth or fifth semester. As mentioned in Chapter 3, ex- 
tending eligibility was allowed for all the cohorts except the last. 

*An omnibus F-test suggests that the baseline characteristics of the first two cohorts differ from the last 
two cohorts (p-value = 0.095). It appears that differences in owning a car and reason for enrolling in college are 
responsible for the moderate p-value. 
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The Opening Doors Demonstration 
Table 4.3 

Educational Outcomes Pre-Katrina, First Through Fourth Semesters: 

First Two Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


First semester 


Registered for any courses (%) 


74.4 


69.8 


4.7 


3.5 


Enrolled full time'' (%) 


60.6 


51.3 


(j 2 ** 


4.1 


Enrolled part time*' (%) 


13.9 


18.4 


-4.6 


3.1 


Number of credits attempted 


7.2 


6.7 


0.5 


0.4 


Regular credits 


4.7 


4.1 


Q *7 ** 


0.3 


Developmental credits 


2.4 


2.6 


-0.2 


0.3 


Number of credits earned 


4.5 


3.4 


^ 2 *** 


0.4 


Regular credits 


3.2 


2.2 


^ Q *** 


0.3 


Developmental credits 


1.4 


1.2 


0.2 


0.2 


Withdrew from one or more courses (%) 


20.7 


27.2 


-6.5 * 


3.5 


Term GPA (%) 


No GPA'' 


32.7 


38.7 


-5.9 


3.8 


Earned a 2.0 GPA or greater 


49.0 


36.9 


12 1 *** 


3.8 


Earned less than a 2.0 GPA 


18.3 


24.4 


-6.2 * 


3.3 


Second semester 


Registered for any courses (%) 


57.1 


38.9 


1 *** 


4.0 


Enrolled full time'' (%) 


45.7 


25.4 


20 3 *** 


4.0 


Enrolled part time*' (%) 


11.4 


13.6 


-2.2 


2.7 


Number of credits attempted 


4.9 


3.5 




0.4 


Regular credits 


3.3 


2.4 


Q ^ *** 


0.3 


Developmental credits 


1.6 


1.1 


0.4 ** 


0.2 


Number of credits earned 


3.4 


2.1 


1 2 *** 


0.3 


Regular credits 


2.4 


1.5 


Q ^ *** 


0.3 


Developmental credits 


1.0 


0.6 


Q 4 *** 


0.1 


Withdrew from one or more courses (%) 


20.8 


16.1 


4.7 


3.2 


Term GPA (%) 


No GPA" 


47.5 


66.6 




4.0 


Earned a 2.0 GPA or greater 


40.4 


19.6 


20.8 *** 


3.7 


Earned less than a 2.0 GPA 


12.2 


13.9 


-1.7 


2.9 



(continued) 
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Table 4.3 (continued) 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Credits earned as of the end of the second semester 


7.9 


5.5 


2 4 *** 


0.6 


Semesters enrolled as of the end of the second semester 


1.3 


1.1 


Q 2 *** 


0.1 


Third semester 


Registered for any courses (%) 


49.4 


37.6 




4.0 


Enrolled full time"* (%) 


31.7 


21.0 


IQ 2 *** 


3.7 


Enrolled part time'’ (%) 


17.7 


16.6 


1.1 


3.2 


Number of credits attempted 


5.4 


4.0 




0.5 


Regular credits 


4.2 


3.0 


1 2 *** 


0.4 


Developmental credits 


1.2 


1.0 


0.2 


0.2 


Number of credits earned 


3.1 


2.2 


Q 9 ** 


0.4 


Regular credits 


2.5 


1.8 


Q 2 ** 


0.3 


Developmental credits 


0.6 


0.4 


0.2 


0.1 


Withdrew from one or more courses (%) 


26.4 


22.1 


4.3 


3.6 


Term GPA (%) 


No GPA‘’ 


57.4 


68.3 


-10 9 *** 


4.0 


Earned a 2.0 GPA or greater 


27.1 


19.2 


2 9** 


3.5 


Earned less than a 2.0 GPA 


15.5 


12.5 


3.0 


3.0 


Credits earned as of the end of the third semester 


11.0 


7.7 


2 3 *** 


0.8 


Semesters enrolled as of the end of the third semester 


1.8 


1.5 


Q 3 *** 


0.1 


Fourth semester 


Registered for any courses (%) 


30.1 


22.9 


y 2 ** 


3.6 


Enrolled full time"' (%) 


17.7 


13.3 


4.4 


3.1 


Enrolled part time'’ (%) 


12.4 


9.6 


2.8 


2.6 


Number of credits attempted 


2.7 


2.1 


0.6 


0.4 


Regular credits 


2.3 


1.7 


0.5 


0.3 


Developmental credits 


0.4 


0.4 


0.0 


0.1 


Number of credits earned 


1.6 


1.5 


0.1 


0.3 


Regular credits 


1.4 


1.2 


0.2 


0.3 


Developmental credits 


0.2 


0.3 


-0.1 


0.1 


Withdrew from one or more courses (%) 


14.3 


7.1 


2 2 *** 


2.5 


Term GPA (%) 


No GPA'’ 


72.5 


77.9 


-5.3 


3.6 


Earned a 2.0 GPA or greater 


19.2 


13.7 


5.5 * 


3.1 


Earned less than a 2.0 GPA 


8.3 


8.5 


-0.2 


2.4 


Credits earned as of the end of the fourth semester 


12.6 


9.2 


3 5 *** 


1.0 


Semesters enrolled as of the end of the fourth semester 


2.1 


1.7 


Q ^ 


0.1 


Sample size (n = 537) 


264 


273 
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Table 4.3 (continued) 

SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College transcript 
data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

^To be considered full-time, students were enrolled in 7 or more credits. 

’’To be considered part-time, students were enrolled in 6 or fewer credits. 

‘’The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 



The GPA outcomes show that the Opening Doors program encouraged more program 
group students to earn an average of “C” or better. That is, it resulted in moving a significant 
number of program students from the lower end of the grade distribution to the higher end (as 
evidenced by the 6 percentage point reduction in GPAs lower than 2.0). 

The second panel of the table shows a large impact on second semester persistence of 
18 percentage points. The outcomes in this panel suggest that the scholarship and counseling 
notably altered credits attempted, credits earned, and GPA among program group members, 
resulting in statistically significant departures from their control group counterparts. Thus far, 
the findings for the first two semesters of the first two cohorts qualitatively mirror the findings 
of the same time period for the entire sample presented above. 

The third and fourth panels of Table 4.3 provide information about the sustainability of 
the behavioral change after the possibility of earning scholarship money had largely ended and 
the counseling was no longer available, at least in theory. The third panel shows that while 
enrollment drops among both groups, program group members are close to 12 percentage points 
more likely to register and close to 1 1 percentage points more likely to register full time. Pro- 
gram group students continue to attempt and earn a greater number of credits when compared 
with their control group counterparts. In addition, program group students continue to earn 
higher grades. The fourth panel shows that program group students are more likely to register 
during the fourth semester after random assignment and they are more likely to attain a GPA of 
2.0 or higher. Program group students are also more likely to withdraw from courses — an out- 
come that could have either a positive or negative interpretation, as noted above. 

Table 4.4 shows enrollment at Delgado and other postsecondary institutions, by stu- 
dents in the first two cohorts who were randomly assigned at Delgado. The pattern of findings is 
qualitatively similar to those reported in Table 4.3 for the full sample. The first panel of the ta- 
ble shows that the Opening Doors program did not have any effect on enrollment at Delgado or 
elsewhere. The second panel shows large enrollment effects at Delgado, qualitatively similar to 
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The Opening Doors Demonstration 
Table 4.4 



Enrollment at Delgado Community College and Other Institutions: 

First Two Cohorts 



Delgado Community College and Louisiana Technical College 



Outcome 


Program 

Group 


Control 

Group Difference 


Standard 

Error 


First semester 

Registered (%) 


At any school 


78.0 


72.3 


5.7 


3.8 


At Delgado Community College 


78.0 


72.3 


5.7 


3.8 


For any course at Delgado (transcript) 


78.0 


72.3 


5.7 


3.8 


At any 2-year school 


78.0 


72.3 


5.7 


3.8 


At any 4-year school 


0.5 


0.0 


0.5 


0.5 


Second semester 

Registered (%) 


At any school 


65.2 


46.4 


18.8 *** 


4.6 


At Delgado Community College 


64.7 


45.4 


2 *** 


4.6 


For any course at Delgado (transcript) 


64.7 


45.5 


IP 2 *** 


4.6 


At any 2-year school 


64.7 


45.9 


18.8 *** 


4.6 


At any 4-year school 


0.5 


0.5 


0.0 


0.7 


As of the end of the second semester 
Registered (%) 


At any school 


87.1 


81.2 


5.9 * 


3.3 


At Delgado Community College 


86.6 


81.2 


5.4 


3.4 


For any course at Delgado (transcript) 


86.1 


80.7 


5.4 


3.4 


Number of semesters enrolled at any school 


1.4 


1.2 


Q 2 *** 


0.1 


Number of semesters enrolled at Delgado (transcript) 


1.4 


1.2 


Q 2 *** 


0.1 


Third semester 

Registered (%) 


At any school 


61.2 


46.0 


15 2 *** 


4.8 


At Delgado Community College 


59.3 


44.2 


15 2 *** 


4.8 


For any course at Delgado (transcript) 


58.8 


44.1 


[4 2 *** 


4.8 


At any 2-year school 


59.3 


44.6 


^4 2 *** 


4.8 


At any 4-year school 


1.9 


1.9 


0.0 


1.3 


As of the end of the third semester 
Registered (%) 


At any school 


88.5 


82.2 


6.4 * 


3.3 


At Delgado Community College 


87.6 


82.2 


5.4 


3.3 


For any course at Delgado (transcript) 


87.1 


82.2 


4.9 


3.3 
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Table 4.4 (continued) 



Outcome (%) 


Program 

Group 


Control 

Group 


Impact 


Standard 

Error 


Number of semesters enrolled at any school 


2.0 


1.6 


Q 4 *** 


0.1 


Number of semesters enrolled at Delgado (transcript) 


2.0 


1.6 


Q 4 *** 


0.1 


Fourth semester 










Registered (%) 










At any school 


35.8 


30.1 


5.8 


4.4 


At Delgado Community College 


34.9 


27.3 


7.6 * 


4.4 


For any course at Delgado (transcript) 


36.8 


27.3 


9.6 ** 


4.4 


At any 2-year school 


35.4 


27.3 


8.1 * 


4.4 


At any 4-year school 


0.5 


2.8 


-2.3 * 


1.2 


As of the end of the fourth semester 










Registered (%) 










At any school 


90.0 


83.6 


6.4 ** 


3.1 


At Delgado Community College 


89.0 


83.1 


5.9 * 


3.1 


For any course at Delgado (transcript) 


88.5 


83.1 


5.4 * 


3.2 


Number of semesters enrolled at any school 


2.4 


1.9 


Q 5 *** 


0.1 


Number of semesters enrolled at Delgado (transcript) 


2.4 


1.9 


Q 5 *** 


0.1 


Sample size (n = 422) 


209 


213 







SOURCES: MDRC calculations using data from the StudentTracker service of the National Student 
Clearinghouse and transcript data for Delgado Community College. 

NOTES: Calculations for this table used all available data for the 422 sample members who were randomly 
assigned at Delgado Community College. 

The Clearinghouse collects data from about 3,100 colleges that enroll 91 percent of U.S. college students 
(http://www.studentclearinghouse.org/about/pdfs/Clearinghouse_profile.pdf). Students have the right to opt 
out of having their information sent. MDRC wasable to find records in the Clearinghouse file for 93 percent of 
the students randomly assigned at Delgado Community College. 

A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Distributions may not add to 100 percent because of rounding. 



those reported in Table 4.2. This panel shows the cumulative effect of registration across the 
first and second semesters. Close to 6 percentage points more Opening Doors students regis- 
tered over the first two semesters. 

The third panel continues to show large effects of the Opening Doors program, with pro- 
gram group students being 15 percentage points more likely to register at any school, including 
Delgado. The pattern changes slightly in the fourth semester: Program group students are more 
likely to register at Delgado (8 or 10 percentage points more likely, depending on whether the 
Clearinghouse or Delgado transcript data are used for inference). Control group students are 
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more likely (by 2.3 percentage points) to transfer to or attend a four-year school. The cumulative 
findings presented in the panel suggest that 90 percent of program group students had registered 
somewhere (“At any school”) as of four semesters after random assignment compared with 84 
percent of control group students, a statistically significant difference of 6 percentage points. 

While these findings are notable because the program was slated to last just the first 
two semesters, it is important to remember that students were permitted to delay their partici- 
pation in the incentive scholarship component. In fact. Table 3.1 in Chapter 3 shows that 
slightly less than 7 percent of students received a scholarship in the third or fourth semester, 
with the percentage being slightly higher in the first two cohorts, as would be expected. It is 
unclear how intensive the program was for such students, or whether the program was more 
intensive for those who used the eligibility consecutively since they may have continued to 
access the Opening Doors counselors. 

The concern about the program being available in some form after two semesters cannot 
be investigated with the current data in this study. For example. Chapter 3 discussed the presence 
of dedicated Opening Doors counselors on each campus, and, in some instances, the counselors 
were the only source of advisement for students. Even though the counseling component was 
more of a monitoring function (counselors largely met with students around the disbursement of 
the scholarship check), survey findings and interviews with administrators in Chapter 3 suggest 
that the counseling stood out as something more meaningful. So while it could be the case that 
the scholarship disbursement largely ceased after the second semester for some groups of stu- 
dents, it could also be the case that the counseling function was sustained in some manner. It is 
plausible, for example, that the program group students made better progress because they had 
greater opportunities to develop relationships with other students, faculty, and perhaps college 
staff outside the Opening Doors program who provided counseling and mentorship. 

In summary, the analysis of later academic outcomes shows that the program continued 
to have impacts as late as the fourth semester after random assignment despite the general de- 
cline in registration over time (as evidenced by the patterns in Figure 4.3). The analysis of just 
the first two cohorts produced findings that were qualitatively similar to those reported for the 
first two semesters of the full sample, suggesting that if more data were available (that is, if Hur- 
ricane Katrina had not occurred), those data would be likely to show similar impacts through the 
fourth semester for the full sample. 



Post-Katrina Outcomes 

Did treatment effects persist even after Hurricane Katrina? All the previous chapters 
have shown ways in which the hurricane affected the study. The effect on enrollment is most 
clearly illustrated in the abmpt decline in enrollment among all cohorts during fall 2005, the 
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academic terni during which the hurricane struck. Despite the severity of the natural disaster, 
Chapter 3 revealed that many students were determined to pick themselves up and start anew. 
This behavior was confirmed in Figure 1.3 in Chapter 1, which showed a dramatic increase in 
enrollment during fall 2006 and subsequent semesters (although not nearly to the pre-Katrina 
enrollment levels). Figure 4.3 also confirms this result using transcript data for the sample 
members in this study. Although very small, the figures show an upturn in registration for the 
semester after Hurricane Katrina (spring 2006) for most cohorts. It is therefore useful to assess 
whether these small changes added to the academic achievement attained by the program group 
prior to Katrina. 

Cumulative Effect on Academic Achievement over Seven Semesters 

Table 4.5 presents the cumulative effect on the Opening Doors program for the first two 
cohorts over seven semesters (or up to 30 months) after random assignment (that is, spring 2004 
through spring 2006). The top panel of the table summarizes the transcript results presented in 
Tables 4.3 and 4.4. It shows that over the first four semesters, program group students were 6.5 
percentage points more likely than control group students to register, and they registered almost 
half a semester more than control group students. Over four semesters, program group students 
earned 3.5 more credits than control group students, and 80 percent of those credits applied to- 
ward a degree or certificate. 

The bottom panel of the table shows the cumulative summary over semesters one 
through seven, where semesters five through seven represent those affected by Hurricane Katri- 
na. The table shows that the post-Katrina semesters do not add much to the story because most 
of the program impacts arise within the first four semesters. Program group students were 5.4 
percentage points more likely to register for any course over the seven semesters since random 
assignment. The small decline in the impact is due to an additional 1 percent of control group 
students registering over semesters five to seven compared with no new registration for program 
group students. However, program group students who registered for classes in semesters one 
through four continued to register and earn credits during semesters five through seven. Pro- 
gram group students earned an additional 1.8 credits in the post-Katrina period (14.4 credits in 
the bottom panel minus the 12.6 credits earned in the pre-Katrina period) compared with control 
group students earning an additional 1.3 credits in the post-Katrina period. Overall, program 
group students were registered for about half a semester longer and earned about 4 more credits 
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The Opening Doors Demonstration 
Table 4.5 



Cumulative Educational Outcomes, First Semester Through Seventh Semester: 

First Two Cohorts 



Delgado Community College and Fouisiana Technical College Report 



Outcome 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Semesters 1 throuah 4 (ore-Katrina) 


Registered for any courses (%) 


84.7 


78.2 


6.5 ** 


3.1 


Number of semesters registered 


2.1 


1.7 


Q 4 *** 


0.1 


Number of credits earned 


12.6 


9.2 


2 2 *** 


1.0 


Regular credits 


9.5 


6.7 


2 g *** 


0.9 


Equated credits 


3.1 


2.5 


0 7** 


0.3 


Cumulative GPA^* (%) 


No GPA” 


22.5 


29.9 


-7.3 ** 


3.6 


Earned a 2.0 GPA or greater 


47.5 


36.5 


J J Q *** 


4.0 


Earned less than a 2.0 GPA 


30.0 


33.7 


-3.7 


4.0 


Semesters 1 throuah 7 


Registered for any courses (%) 


84.7 


79.3 


5.4 * 


3.1 


Number of semesters registered 


2.5 


2.0 


0 5 *** 


0.1 


Number of credits earned 


14.4 


10.5 


2 9 *** 


1.3 


Regular credits 


11.2 


7.9 


^ 2 *** 


1.1 


Equated credits 


3.2 


2.6 


0.6 * 


0.3 


Cumulative GPA^ (%) 


No GPA'’ 


22.5 


28.0 


-5.5 


3.6 


Earned a 2.0 GPA or greater 


45.6 


36.9 


g y ** 


4.0 


Earned less than a 2.0 GPA 


31.9 


35.1 


-3.3 


4.1 


Sample size (n = 537) 


264 


273 







SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College transcript 
data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“Cumulative GPA is based on credit-bearing courses taken from random assignment through the end of the 
third postprogram semester. Courses in which students did not receive a passing grade and subsequently 
repeated are not included in the cumulative GPA. 

'’The "No GPA" category includes students who did not enroll and students who took only developmental 
courses, which are not included in GPA calculations. 
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than their control group counterparts, suggesting a net gain of four-tenths of a credit over 
semesters five through seven.^ In summary, most of the effects reported in this chapter oc- 
curred during the pre-Katrina period, with little effects evident in the post-Katrina period. 

Effect on Students’ Level of Effort 

The previous analyses suggest that the Opening Doors program increased the academic 
achievement of students. An important question is why this increase might have occurred. Spe- 
cifically, the incentive built into the scholarship would be expected to have motivated students 
to focus more on their studies by, for example, spending time on class assignments and com- 
pleting those assignments. To gauge whether program students put more time and effort into 
their studies as a result of the intervention, outcomes from the student survey are presented in 
Table 4.6. The top panel of the table shows 12-month impacts on the level of effort. That is, the 
data represent students’ responses to questions about their level of effort during the first two pro- 
gram semesters.'” Table 4.6 shows that program group students and control group students re- 
ported spending a similar number of hours on campus and studying during the first semester. 
However, program group students spent close to 1 hour more on campus and studying than con- 
trol group students reported spending during the second semester, which could reflect, in part, the 
fact that program group students were more likely to register for classes for the second semester. 

Similar to the cumulative registration rates reported in Table 4.5, the bottom panel of 
Table 4.6 shows that about 5 percentage points more program group students than control group 
students reported attending classes since random assignment, an increase of about 6 percent. 

Effect on Employment Outcomes 

Another way that the intervention could have helped students focus more on their studies 
is by inducing them to spend less time in paid employment while attending school. While such 
an inducement is a potentially important mechanism, enrollment dates cannot be easily matched 
with respondent reports of employment in order to determine whether students changed their 

^Appendix Table C.3 shows a similar analysis using all cohorts. While the impacts are somewhat smaller 
(largely reflecting the greater number of post-Katrina semesters for the third and fourth cohorts), they are quali- 
tatively similar to those reported in Table 4.5. Overall, a larger proportion of program group students registered 
for classes in the post-Katrina period, indicating that this enrollment occurred mainly among the fall 2004 and 
spring 2005 cohorts. In fact, program group students earned about 5.2 credits in the post-Katrina period com- 
pared with 4.6 credits earned by the control group students, for a gain of six-tenths of a credit earned in the 
post-Katrina period (equal to the post-Katrina impact of 2.9 for number of credits earned minus the pre-Katrina 
impact of 2.3 credits). Appendix Tables C.4 and C.5 repeat this analysis for Delgado Community College only, 
using Clearinghouse and transcript data, respectively. 

'"The responses were collected from 12 to 18 months after random assignment because of Hurricane Ka- 
trina. See Chapter 2 for a detailed discussion of the survey sample. 
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The Opening Doors Demonstration 
Table 4.6 

Level of Academic Effort and Frequency of Attendance: Survey Responses 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Number of 
Observations 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Level of academic effort 

Hours spent 

On campus in first semester 


789 


10.0 


9.4 


0.6 


0.4 


Studying in first semester 


791 


7.5 


6.9 


0.5 


0.4 


On campus in second semester 


792 


7.2 


6.2 


1.0 ** 


0.5 


Studying in second semester 


792 


5.6 


4.6 


1.0 ** 


0.4 


Freauencv of attendance 

Attended classes since random 
assignment (%) 


796 


82.9 


78.4 


4.6 * 


2.6 



SOURCE: MDRC calculations using the Opening Doors 12-Month Survey. 

NOTES: See Chapter 2 for a discussion of the follow-up sample and an explanation of sample sizes. 
Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 



work hours while enrolled or whether these activities occurred at different times. Nevertheless, 
analyses of survey data on the number of jobs worked and usual hours worked (not shown) sug- 
gest that assignment to the program did not affect the decision to work while in school. 



Discussion of Findings 

This chapter presented evidence that the Opening Doors program, which combined a 
generous incentive scholarship program with counseling, resulted in greater academic achieve- 
ment during the semesters in which the program operated as well as the following semesters 
when the program was not, for the most part, in place. While the analysis of the follow-up pe- 
riod is suggestive that the intervention’s effects were sustainable (although there is some attenu- 
ation), evidence for this effect is limited by the influence of Hurricane Katrina, which signifi- 
cantly intermpted the lives of students and the operations of the institutions in the study. Two 
key findings appear to be: 
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• The program resulted in increased second semester enrollment by 15 per- 
centage points (an increase of 30 percent), which seems to be larger when 
the semester occurs in the summer (by 25 percentage points, or an increase 
of about 85 percent)." 

• Positive effects on academic outcomes are evident as late as the fourth 
semester after random assignment. 

However, while the percentage differences reported in this chapter are large, an actual credit 
achievement of 3.0 credits over four semesters is not a cause for celebration. As discussed in the 
chapter, despite the analysis of seven semesters of data, the bulk of the credit gain was achieved 
prior to Hurricane Katrina. By itself, this amount of educational attainment is unlikely to trans- 
late into meaningful employment and earnings differences. It is important to know whether such 
gains would have continued or translated into the acquisition of certificates or associate’s de- 
grees after a period of time. 



"while not directly comparable to this retention estimate, Bettinger (2004) finds that a $1,000 increase in 
Pell Grant eligibility increases persistence between the first and second years of college attendance by 2 to 4 
percentage points. Dynarski (2005) finds that merit aid of about $3,000 increases the probability of persistence 
by 5 percent to 1 1 percent among those who would have gone to college in the absence of the financial aid. 
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Chapter 5 

Social, Psychological, and Health Outcomes 



Community college reforms like the Opening Doors program at Delgado Community 
College and Louisiana Technical College- West Jefferson (LTC), which offered scholarships with 
counseling, may come to affect the well-being of participating students over time. “Well-being” 
is broadly defined to include a range of social, psychological, and health outcomes. This chapter 
examines whether this program had any effects on well-being among the Louisiana sample at the 
time of the follow-up survey, which is detailed in Chapter 2. It begins with brief discussions of 
the Opening Doors conceptual framework and relevant research on the relationship between 
education and health. The chapter then discusses the analytic sample and presents baseline statis- 
tics on well-being. This is followed by findings on the impact of the Opening Doors program on 
social, psychological, and health outcomes. The chapter concludes with a brief discussion of the 
implications of these findings for future studies of community college reforms and student well- 
being. Finally, in addition to its human toll, the large-scale tragedy of Hurricane Katrina created a 
challenge to our efforts to uncover program impacts on these outcomes, as also discussed below. 

In sum, the key findings from these analyses are: 

• At the start of the study, students in the Louisiana sample exhibited a gener- 
ally good sense of well-being, presumably due in large part to the fact that 
the sample is largely made up of young adults. However, they also demon- 
strated relatively high levels of psychological distress, suggesting that there 
might be room for significant impacts on their social and psychological well- 
being over time. 

• Analyses based on the follow-up survey indicate that the Opening Doors 
program had positive impacts on a range of social and psychological out- 
comes. It promoted greater levels of engagement in the pursuit of personal 
goals and in life more generally. Moreover, these results suggest positive 
program effects on self-esteem and overall sense of self Finally, the program 
appears to have engendered a greater sense of perceived social support and 
contributed to more political engagement. 

• There is no evidence that the Opening Doors program had a beneficial effect 
on mental and physical health outcomes. The members of the program and 
control groups did not have statistically significant differences in rates of 
smoking, perceived stress, psychological distress, self-rated health, or body 
mass index (BMI). 
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Opening Doors and Well-Being 

The fundamental goal of Opening Doors is to help community college students achieve 
better educational outcomes, like persistence, better grades, and degree completion. Some of 
these effects will be achieved in the near term (for example, in the first semester of the interven- 
tion), and others will be realized only with the passage of time (for instance, in subsequent 
semesters and after graduation). 

Early program impacts on educational outcomes — especially if they are strong or endur- 
ing — have the potential to influence not only educational attainment but also labor market out- 
comes (like employment, better jobs, and higher income) and indicators of well-being in the long- 
er term. In sum. Opening Doors can act as a lever that helps students capitalize on the benefits of 
education and enjoy greater socioeconomic status (SES) and subsequently an enhanced sense of 
well-being.' (SES is a multidimensional constmct reflecting a person’s position in the class stmc- 
ture of a society. SES is typically assessed with measures of educational attainment, income, and 
occupational status or prestige, all of which are important to studies of health.)^ The indicators of 
well-being examined in this chapter include positive social, psychological, and health outcomes, 
such as having a stronger sense of self, better interpersonal relationships and greater social sup- 
ports, more civic engagement and pro-social behavior, greater capacity to manage stress, healthier 
attitudes and behaviors, and — ultimately — better physical and mental health. 

Although these outcomes are of great interest, it is sensible to have low expectations 
about the degree to which the Opening Doors program in Eouisiana might affect them during this 
follow-up period. As shown in the conceptual framework that guides the Opening Doors demon- 
stration (Chapter 1, Figure 1.2), the Opening Doors interventions are theorized to have the great- 
est potential for influencing well-being through indirect effects, which emanate from a direct im- 
pact of the intervention on educational outcomes, which in turn can lead to higher SES. As dis- 
cussed below, SES has consistently been shown to be a powerful determinant of well-being. This 
hypothesized causal chain of effects cannot frilly evolve in the follow-up period under observa- 
tion here, although the earliest signs of it might be evident. Moreover, even direct effects of the 
program on well-being — for example, positive changes in students’ self-esteem or peer net- 
works stemming directly from their experience of the intervention — may also require consider- 
able time before leading to observable changes in health behaviors. Eikewise, changes in health 
behaviors typically do not lead to immediate effects on health. Finally, Hurricane Katrina and its 
aftermath have adversely affected the lives of Opening Doors participants in ways that compli- 
cate our efforts to discern program impacts on well-being, as discussed further below. 



'it is also possible that some community college reforms may have direct and independent effects on long- 
er term outcomes, and these relationships are also examined. 

^See Cockerham (2007:63). 
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Therefore, expectations of finding impacts on well-being should not be high. However, 
some outcomes may be more likely to change within the follow-up period of the survey than 
others. Therefore, three general kinds of well-being are distinguished and examined: (1) social 
and psychological well-being; (2) health behaviors; and (3) mental and physical health. As dis- 
cussed above, outcomes in the first two categories are more susceptible to change than those in 
the third category, particularly within this time frame. 



Relevant Research 

The most compelling evidence to support the theorized relationships discussed above 
comes from the long-standing research literature that documents a strong positive association be- 
tween education and health. Individuals with higher levels of education have lower mortality and 
morbidity rates from many types of disease and display better health habits.^ For example, they 
demonstrate lower levels of smoking and binge drinking, and have a lower prevalence of obesity.'* 

However, it is important to emphasize that although there is clear evidence that higher 
levels of education are associated with enhanced well-being,^ it is not clear whether this asso- 
ciation represents a causal effect of education on these outcomes. Instead, it could be that fac- 
tors leading to greater success in school and higher educational attainment — such as having a 
wealthier or more supportive family, or better health in childhood — account for the observed 
relationship. The Opening Doors study is especially important since, because of its randomized 
design, it can provide information on whether improvements in educational opportunities can 
indeed improve social, psychological, and health outcomes. In short, to the extent to which the 
Opening Doors program has early impacts on educational outcomes, it has the potential to em- 
pirically demonstrate the subsequent effects of education on well-being in the longer term. 

This focal relationship between education and health must also be understood as part of 
a more fundamental relationship between SES and health,*’ which is especially relevant to any 
discussion of the well-being of low-income populations. Research has consistently identified an 
“SES health gradient,”^ wherein improvements in components of SES — such as social class, 
education, income, and occupational prestige — are associated with better health. Therefore, 
although education — among the indicators of SES — may be the most significant determinant 



^For studies linking higher levels of edueation with lower mortality, see Lleras-Muney (2005); Christen- 
son and Johnson (1995); Deaton and Paxson (2001); and Elo and Preston (1996). 

'*Cutler and Lleras-Muney (2006). 

^Kolata (2007). 

*’Link and Phelan (1995). 

^Adler and Newman (2002); Marmot and Wilkinson (1999). 
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of well-being,® it should be viewed as part of this larger construct of SES.^ This is important to 
consider given Opening Doors’ focus on low-income students, who — according to all measures 
of SES — have considerable ground to gain. 

In addition, education is believed to foster a greater sense of self and connectivity to 
others.'® For example, greater educational attainment can strengthen and expand the social 
networks from which one can draw guidance, help, and support. It can also bring about greater 
awareness of social issues and civic engagement. Colleges and universities provide students 
with access to social support through relationships with fellow students, faculty, and staff," 
some of whom are likely to become role models and mentors, and studies show that students 
who have mentors may have a greater sense of self-worth, be better able to weather personal 
crises, become more aware of educational and career opportunities, and ultimately set higher 
goals for themselves.'^ Moreover, to the extent that institutions successfully create a supportive 
and “bonding” environment, students — younger students especially — may be less likely to 
engage in behaviors that compromise their own and their community’s well-being.'® In addi- 
tion, studies illustrate that educational attainment is positively associated with broader social 
involvements, as indicated by effects on voting, volunteerism, newspaper readership, civic 
knowledge, and involvement in social groups and clubs. ''' Thus, to the degree to which the 
Opening Doors program influences early educational outcomes, it may also affect some social 
and psychological indicators of well-being in the follow-up period. 



Findings 

This chapter focuses on social, psychological, and health outcomes among participants 
in the Opening Doors program. However, lessons from earlier analyses help to frame the find- 
ings presented here. 

As reported in Chapters 3 and 4, the Opening Doors program significantly influenced 
students’ experiences and progress at Delgado and ETC. It led to greater contact with college 
staff and faculty for advice and support with personal matters. In particular, the program partici- 



®Ross and Wu (1995). 

^Recent research suggests that educahon is more predictive of the onset of health problems, while income 
is more strongly associated with the progression of such problems (Herd, Goesling, and House, 2007). 

'®See Berkman and Glass (2000) and House (2001) for research and commentary on the deleterious ef- 
fects of social isolation on health. 

"a large body of research suggests that people who enjoy strong and supportive relationships with others 
are better able to cope with stressful life events and circumstances and, consequently, to preserve their emo- 
tional and physical well-being (Thoits, 1995; Turner and Turner, 1999). 

'"Rhodes (2002). 

'®Eccles and Gootman (2001); Neumark-Sztainer, Story, French, and Resnick (1997); Resnick et al. (1997). 

'“'Dee (2004); Belli Carpini and Keeter (1996); Sullivan and Transue (1999); Uslaner, (2002). 
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pants reported greater access to a mentor at the college, someone from whom they could seek 
guidance regarding school- or work-related matters. In addition, the program had positive ef- 
fects on a range of educational outcomes, including persistence and credits earned. These im- 
pacts on the duration and quality of the educational experience — and ultimately on the like- 
lihood of educational success — indicate the potential for program impacts on more distal out- 
comes, such as well-being. 

Moreover, as discussed below, the scholarships — the “extra money” in and of itself — 
offered by the program held the potential to directly enhance a sense of well-being and perception 
of support, particularly among students such as these who experience great economic hardship. 



Measures of Well-Being 

The measures of well-being examined in this chapter can be grouped into two general 
categories: (1) social and psychological outcomes, and (2) health outcomes, as illustrated below. 
The latter category includes health behaviors as well as indicators of physical and mental health. 
With regard to health behaviors, just one critically important behavior, smoking, is examined. 
The follow-up survey also collected information on other health behaviors, including binge 
drinking, illegal dmg use, and risky sexual behavior. However, very few members of the sample 
reported having engaged in those behaviors at baseline or follow-up. 

Social and psychological outcomes, the first category, comprise the following elements 
in this study: 

• Outlook and identity: optimism, goal engagement, life engagement, self- 
esteem, and sense of self 

• Social support and civic engagement: general social support, volunteerism, 
political engagement, and having friends who value education 

• Antisocial behavior: having spent time in reform school or prison (either the 
respondent or a good friend) 

Health outcomes in this study comprise the following elements: 

• Perceived stress, psychological distress,'^ self-rated health, BMI (which is 
based on a calculation using one’s height and weight), and smoking 



Scale measures of “nonspecific psychological distress” are designed to assess or “screen for” a range of 
broadly defined mental disorders — such as mood or anxiety disorders — rather than any one disorder in par- 
ticular (for example, major depression) (Kessler et al., 2002; Furukawa, Kessler, Slade, and Andrews, 2003). 
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Data for these indicators of well-being were collected via the follow-up survey, and da- 
ta for a subset of these indicators were collected at baseline using a briefer survey instrument. 
Some of these variables are represented with scale measures (variables that respondents are 
asked to rate on a continuum, or scale, of numerical values — such as 1 to 4), which are sum- 
marized in Box 5.1 and detailed in Appendix D. All other variable measures are described in 
footnotes to the tables that follow. 



Survey Sample 

As illustrated in Chapter 2, Figure 2.3, Hurricane Katrina shuck Louisiana in the midst 
of the fieldwork to collect follow-up data for the evaluation of the Opening Doors program. Con- 
sequently, some study participants responded to the follow-up questionnaire prior to the hurri- 
cane (Group A, n = 492), and some responded afterwards (Group B, n = 309). The analyses re- 
ported below are carried out using this combined sample of people who completed the follow-up 
survey, and is henceforth referred to as the survey sample (N = 801). Alternative analytic strate- 
gies treating the two groups (A and B) separately are discussed at the end of this chapter. 



Baseline Indicators of Social and Psychological Well-Being and 
Health 

The Opening Doors program was offered to parents with children under age 19 whose 
family incomes were below 200 percent of the federal poverty level. As detailed in Chapter 2, 
the study sample is overwhelmingly female and black. At baseline about half were currently 
employed, and the average age was 25. Although the sample is somewhat older than traditional 
college student populations, it is nonetheless a “young aduft” sample. Therefore, this sample 
generally comprises young black women, many of whom work in addition to going to school, 
and all of whom have parenting responsibilities. 

Table 5.1 presents baseline measures of well-being for the survey sample.''’ As shown 
in the table, on the four-point scale assessing general social support, the average rating is 3.2, 
where 3 equals “agree” and 4 equals “strongly agree” with statements indicating the presence 
of support. The average rating on the five-point scale that measures perceived stress was 2.4, 
where 2 equals “almost never” and 3 equals “sometimes.” Therefore, levels of perceived 



'^This table, as well as the other tables in this chapter, follows the same format that was used in previous 
chapters. However, it also includes the “effect size” for each outcome, defined as the difference between the 
outcome for the program and control groups divided by the standard deviation for the control group. Dividing 
by the standard deviation “normalizes” the program’s impact so that it is expressed in standard deviation units. 
This standardization makes it easier to compare program impacts across outcomes that are measured using 
different scales. 
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Box 5.1 



Behind the Scales Measuring Well-Being 

Outlook and Identity 

Optimism. To assess respondents’ level of optimism, they were asked to indicate the degree to which they 
agreed (on a scale of 1-4) with the following types of statements: In uncertain times, I usually expect the 
best, and, Overall, I expect more good things to happen to me than bad. 

Goal engagement. To gauge respondents’ level of engagement in working toward their goals, they were 
asked to indicate the degree to which they agreed (on a scale of 1-4) with the following types of statements: 
I don ’t think much about my long-term goals; and. It is important for me to take time to plan out where I’m 
going in life. 

Life engagement. To gauge respondents’ level of engagement in life in general, they were asked to indicate 
the degree to which they agreed (on a scale of 1 -4) with the following types of statements: / have lots of rea- 
sons for living-, and, I value my activities a lot. 

Self-esteem. To measure respondents’ level of self-esteem, they were asked to indicate the degree to which 
they agreed (on a scale of 1 -4) with the following types of statements: lam able to do things as well as most 
other people-, and, / feel that I have a number of good qualities. 

Sense of self. To measure the degree to which respondents have a well developed sense of themselves as in- 
dividuals and in relation to their significant others, they were asked to indicate the degree to which they 
agreed (on a scale of 1-4) with the following types of statements: There is at least one person who knows 
“the real you You have a pretty good sense of the path you want to take in life and the steps to take to get 

there; You feel your life is filled with meaning, a sense of purpose; and. People often seek your advice and 
support. 

Social Support and Civic Engagement 

General soeial support. To assess respondents’ perceived level of social support, they were asked to indi- 
cate the degree to which they agreed (on a scale of 1-4) with the following types of statements: There are 
people I know will help me if I need it; and, I have a trustworthy person to turn to if I have problems. 

Friends value edueation. To gauge the degree to which respondents have friends who value educational 
pursuits, they were asked to indicate the extent to which they felt their friends supported their efforts to 
achieve educational goals as well as had educational goals of their own (on a scale of 1-4). This was done 
with the following types of statements: Among your friends, how important is it to go to college?; and. How 
important is it to get good grades? 

Political engagement. To measure respondents’ level of civic engagement, they were asked to indicate (by 
responding “yes” or “no”) whether they had done things like registered to vote and donated time or money 
to a political campaign. 

Mental Health Outcomes 

Stress. To measure respondents’ level of stress, they were asked to indicate how often (on a scale of 1-5) 
they felt, for example, that they were unable to control the important things in life; and that difficulties were 
piling up so high [that you] could not overcome them. 

Psychological distress. To measure respondents’ level of psychological distress, they were asked to indicate 
how often (on a scale of 0-4) they felt, for example, hopeless-, restless or fidgety-, and worthless. 
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The Opening Doors Demonstration 
Table 5.1 



Social, Psychological, and Health Measures of Sample Members at Baseline 
Delgado Community College and Louisiana Technical College Report 



Outcome 


Sample 

Size 


Full 

Sample 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


General social support'* (1-4) 


769 


3.20 


3.20 


3.20 


0.00 


0.03 


0.01 


Perceived stress’* (1-5) 


783 


2.39 


2.32 


2.46 


-0.15 ** 


0.06 


-0.18 


K6 score for psychological 
distress'* (0-24) 


766 


4.95 


5.03 


4.88 


0.15 


0.29 


0.04 


Indicator of high 
psychological distress'* (%) 


766 


5.5 


5.5 


5.4 


0.1 


1.6 


0.0 


Health status fair or poor (%) 


786 


3.6 


3.6 


3.6 


0.0 


1.3 


0.0 


Body mass index** (kg/m^) 


778 


28.49 


28.94 


28.04 


0.90 * 


0.51 


0.13 


Overweight or obese 
(BMI>25)‘'(%) 


778 


64.0 


66.7 


61.3 


5.4 


3.4 


0.1 


Current smoker (%) 


777 


9.4 


8.2 


10.6 


-2.5 


2.1 


-0.1 



SOURCE: MDRC calculations from the Opening Doors Baseline Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

'“An 8-item scale about the presence of social support; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." Items are averaged. 

'’A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very often." 
Items are averaged. 

‘^A 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

■^An indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 

“^BMI = weight in kilograms divided by height in meters squared. 

^1 = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight less than 18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and obese 
= 30 or greater. 
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social support are fairly high, and levels of perceived stress are fairly low, at baseline. However, 
about 5.5 percent meet the criteria for experiencing serious psychological distress, which is 
nearly twice as high as the average for the general U.S. population (3.1 percent).'’ Therefore, 
despite their youth and resilience, these students also demonstrate relatively high levels of psy- 
chological distress, suggesting that there might be room for significant impacts on their social 
and psychological well-being over time. 

As also shown in the table, less than 4 percent rate their overall health as “fair” or 
“poor,” meaning that the vast majority stated their health is “good,” “very good,” or “excel- 
lent” at baseline. Nearly two-thirds (64.0 percent) meet BMI criteria for being “overweight” or 
“obese,” which is similar to the average for the general adult population nationally (66 per- 
cent).'* Overall, at the start of the study, students in the study sample exhibit generally good 
health, although — given that they are typically young adults — their tendency to be over- 
weight or obese is troubling. 

There are two significant baseline differences in well-being between the program and 
control groups. The program group reports slightly lower perceived stress and slightly higher 
BMI than the control group does, and these differences are addressed in the analyses that fol- 
low. In addition, there are three baseline differences between those who responded to the fol- 
low-up survey and those who did not (that is, those who were lost to follow-up). As shown in 
Appendix Table E.l, individuals reporting better health (that is, those who were less likely to 
rate their health as “fair” or “poor”) and a marginally lower BMI at baseline are less likely to 
have completed the follow-up questionnaire. Moreover, “current smokers” at baseline were 
significantly — and considerably — less likely to have been reached for follow-up. These dif- 
ferences indicate possible selective attrition, which could lead to biased estimates of the pro- 
gram effects. Consequently, it was important to control for baseline characteristics to the extent 
possible in the impact analyses that follow, as detailed below. 



Impacts on Social, Psychological, and Health Outcomes 

As shown in Table 5.2, analyses of data from the follow-up survey produced evidence 
to suggest that the Opening Doors program had positive impacts on a range of social and psy- 
chological outcomes — promoting greater levels of engagement in the pursuit of personal goals 
and in life more generally. Moreover, these results suggest positive program effects on two 
key aspects of identity, self-esteem and overall sense of self. Finally, the program appears to 



'’Based on results from the National Health Interview Survey (National Center for Health Statistics, 
2005). Web site: www.cdc.gov/nchs/about/major/nhis/released200506.htm. 

'*Based on results from the National Health and Nutrition Examination Survey (National Center for 
Health Statistics, 2007). Web site: www.cdc.gov/nccdphp/dnpa/obesity/frend/index.htm. 
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The Opening Doors Demonstration 
Table 5.2 

Social and Psychological Outcomes 
Delgado Community College and Louisiana Technical College Report 



Average Scale Points 

Sample Full Program Control Standard Effect 



Outcome 


Size Sample 


Group 


Group Difference 


Error 


Size 


Outlook and identity 


Optimism“(l-4) 


794 


3.11 


3.12 


3.09 


0.03 


0.03 


0.06 


Goal engagement'’ (1-4) 


795 


3.60 


3.65 


3.55 


0.10 *** 


0.03 


0.21 


Life engagement (1-4) 


795 


3.57 


3.59 


3.54 


0.06 * 


0.03 


0.13 


Self-esteem'' (1-4) 


796 


3.57 


3.60 


3.54 


0.06 ** 


0.03 


0.15 


Sense of selt(l-4) 


794 


3.61 


3.64 


3.58 


0.06 ** 


0.03 


0.16 


Social suDDort and civic engaBcment 


General social support^ (1-4) 


792 


3.25 


3.30 


3.20 


0.10 *** 


0.03 


0.21 


Friends value education® (1-4) 


773 


3.22 


3.19 


3.25 


-0.06 


0.07 


-0.07 


Volunteerism’’ (%) 


799 


25.4 


26.7 


24.1 


2.6 


3.1 


0.1 


Political engagement' (0-1) 


797 


0.44 


0.46 


0.43 


0.03 ** 


0.01 


0.14 


Antisocial behavior in oast year 


Spent time in reform school or prison (%) 
Close friend spent time in reform 


799 


4.5 


4.5 


4.5 


0.1 


1.5 


0.0 


school or prison (%) 


799 


24.4 


25.9 


22.9 


3.1 


3.0 


0.1 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“A 6-item scale about feelings of optimism; response categories range from 1 = "strongly disagree" to 4 = 
"strongly agree." Items are averaged. 

'^A 3-item scale about engagement in working toward their goals; response categories range from 1 = "strongly 
disagree" to 4 = "strongly agree." Items are averaged. 

■^A 6-item scale about engagement in life in general; response categories range from 1 = "strongly disagree" to 
4 = "strongly agree." Items are averaged. 

■*A 4-item scale about feelings of self-esteem; response categories range from 1 = "strongly disagree" to 4 = 
"strongly agree." Items are averaged. 

=A 13-item scale about their sense of themselves as individuals and in relation to their significant others; 
response categories range from 1 = "strongly disagree" to 4 = "strongly agree." Items are averaged. 

*An 8-item scale about the presence of social support; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." Items are averaged. 

EA 5-item scale about the degree to which their friends value educational pursuits; response categories range 
from 1 = "not very" to 4 = "extremely." Items are averaged. 

•'Performed any unpaid volunteer or community service work since random assignment. 

‘A 4-item summative scale about their level of political engagement; response categories are 1 = "yes" and 0 = 
"no." The foiu- items are added together and divided by four. 
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have engendered a greater sense of perceived social support and contributed to more political 
engagement. These impacts are illustrated in the table in terms of group differences and effect 
sizes. According to the latter, the largest program impacts were on social support and goal en- 
gagement. Although it is difficult to translate what these individual impacts “mean” in practical 
terms, it is nonetheless encouraging to observe the consistently positive pattern of results across 
this range of indicators. In contrast, there were no program impacts on physical or mental 
health, or on the likelihood of becoming a “current smoker,” as shown in Table 5.3. 

It is common, in studies of the effects of programs on health outcomes, to control for 
health at baseline. It was important to do so in this case given the differences in baseline values 
of perceived stress and BMI reported in Table 5.1. Table 5.4 contains additional analyses that 
examine impacts on a subset of social, psychological, and health outcomes, controlling for the 
baseline values of those outcomes. The outcomes presented here are those for which both base- 
line and follow-up assessments were made. These analyses produced additional evidence to 
suggest that the Opening Doors program had a positive effect on perceived social support. 
However, controlling for baseline health outcomes does not alter the conclusion that the pro- 
gram had no effect on physical or mental health outcomes. 



Implications for Future Research 

Overall, these analyses of the Opening Doors survey sample suggest that the Opening 
Doors program not only brought about positive academic outcomes, but also contributed to en- 
hanced social and psychological well-being. There are two main interpretations of these find- 
ings. First — as illustrated in the conceptual model that guides the Opening Doors demonstra- 
tion (Chapter 1, Figure 1.1) — these social and psychological benefits may emerge primarily 
indirectly via the program’s more proximate impacts on early educational outcomes. In other 
words, short-term educational successes may be acting as a critical lever that sets in motion the 
development of an enhanced sense of well-being, in particular a heightened sense of who one is 
or wants to be, perceived support from others, and connections to community (in the form of 
political involvement). It may be, for example, that as students in the program group progressed 
in school, they simultaneously had greater opportunities to develop relationships with other stu- 
dents, faculty, and perhaps especially with college staff who provided counseling and mentor- 
ing. In this way, the program may have facilitated the development of a more supportive college 
environment. It may be useful for future studies of community-college programs to further ex- 
amine how different relationships within the campus context can be uniquely supportive. 
Second, it could also be that — to some extent — these impacts on social and psychological 
well-being may have been direct results of the scholarship itself — the “extra money,” per se. 
Among students from impoverished backgrounds, this money may have been interpreted as a 
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The Opening Doors Demonstration 
Table 5.3 



Health Outcomes 

Delgado Community College and Louisiana Technical College Report 



Average Scale Points 



Outcome 


Sample 

Size 


Full 

Sample 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


Perceived stress^ (1-5) 


796 


2.13 


2.11 


2.15 


-0.04 


0.06 


-0.04 


K6 score for psychological 
distress'’ (0-24) 


796 


5.66 


5.61 


5.72 


-0.11 


0.31 


-0.03 


Indicator of high psychological 
distress” (%) 


796 


7.9 


7.7 


8.1 


-0.4 


1.9 


0.0 


Health status fair or poor (%) 


798 


14.5 


14.2 


14.9 


-0.6 


2.5 


0.0 


Body mass index”* (kg/m^) 


745 


28.0 


28.1 


27.8 


0.4 


0.5 


0.1 


Overweight or obese (BMI > 25)” (%) 


745 


63.4 


65.5 


61.3 


4.1 


3.5 


0.1 


Current smoker (%) 


799 


15.0 


14.7 


15.3 


-0.6 


2.5 


0.0 



SOURCE: MDRC calculations from the Opening Doors 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

'“A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very 
often." Items are averaged. 

'’A 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

*^An indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 

‘'BMI = weight in kilograms divided by height in meters squared. 

*^1 = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight is less than 18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and 
obese = 30 or greater. 



powerful sign that “someone was investing in me.” Future analyses of similar interventions can 
further address these and other interpretations. 



Alternative Analysis Samples 

As mentioned above and in Chapter 2 (Figure 2.3), the follow-up sample contains two 
groups: Group A, which consists of respondents who completed the follow-up survey before 
Hurricane Katrina, and Group B, which consists of respondents who completed the follow-up 
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The Opening Doors Demonstration 
Table 5.4 



Social, Psychological, and Health Outcomes: 

Controlling for Baseline Value of Outcomes 

Delgado Community College and Louisiana Technical College Report 



Outcome 


Sample 

Size 


Full 

Sample 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Effect 

Size 


General social support'* (1-4) 


755 


3.26 


3.32 


3.21 


Q 1 1 *** 


0.03 


0.24 


Perceived stress’* (1-5) 


778 


2.12 


2.13 


2.11 


0.01 


0.06 


0.01 


K6 score for psychological 
distress'* (0-24) 


761 


5.66 


5.65 


5.68 


-0.03 


0.29 


-0.01 


Indicator of high 
psychological distress'* (%) 


761 


7.8 


7.6 


7.9 


-0.2 


1.9 


0.0 


Health status fair or poor (%) 


783 


14.4 


14.2 


14.7 


-0.5 


2.4 


0.0 


Body mass index" (kg/m^) 


742 


27.95 


27.85 


28.05 


-0.20 


0.24 


-0.03 


Overweight or obese 
(BMI>25)‘'(%) 


742 


63.2 


63.4 


63.1 


0.3 


2.2 


0.0 


Current Smoker (%) 


775 


14.6 


14.8 


14.3 


0.5 


1.9 


0.0 



SOURCES: MDRC calculations from the Opening Doors Baseline Survey and 12-Month Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort, campus, and the value at baseline. 

'“An 8-item scale about the presence of social support; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." Items are averaged. 

'’A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very often." 
Items are averaged. 

‘^A 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

■^An indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 

*^BMI = weight in kilograms divided by height in meters squared. 

T = BMI of25 or greater; 0 = BMI ofless than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight less than 18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and obese 
= 30 or greater. 



survey after the hurricane ended. The program effects might be expected to differ across these 
two groups. The participants in Group A conducted the follow-up survey sooner after enroll- 
ment than did those in Group B, and they had not been exposed to the hurricane at the time of 
follow-up. The overwhelming effects of the hurricane may have washed out any positive pro- 
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gram effects for those in Group B. Furthermore, even putting aside the hurricane, program ef- 
fects in Group B could be smaller if the benefits of the program decay with time. On the other 
hand, the hurricane could have amplified any positive program effects if the program benefits 
improved the participants’ ability to cope with adversities resulting from the storm and subse- 
quent flood. If so, the program effects could be larger among those in Group B than in Group A. 

For these reasons, it may seem reasonable to assess the program effects separately 
across the two groups. Estimates of program effects for those in Group A may be of particular 
interest, since they are not contaminated by the hurricane. However, it should be noted that even 
results for Group A alone will not provide an accurate picture of the program effects if members 
of Groups A and B differed in ways that influenced the effectiveness of the program. There are 
reasons to think that these groups did differ along several dimensions. As shown in Appendix 
Table E.2, there were significant differences between Groups A and B at baseline: Those inter- 
viewed before the storm and flood had significantly higher levels of social support and lower 
levels of perceived stress and psychological distress at baseline. To an unknown degree, the 
benefits of the Opening Doors program may have been larger for the members of this group, 
who had yet to experience the hurricane and its aftermath. 

Estimates of the program impacts for Groups A and B (Appendix Table E.3) indicate 
that Group A is driving most of the effects reported above, although the general pattern of re- 
sults is consistent across Groups A and B. Furthermore, members of the program group for both 
Groups A and B show significant program effects on perceived social support. Although this 
analysis uncovers more program impacts among Group A than among Group B, it is not possi- 
ble to determine the degree to which this difference came about because (1) Hurricane Katrina 
reduced the program’s effects, (2) the program’s effects naturally diminished over time, or that 
(3) members of Group A had characteristics that led to larger program effects. 
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Chapter 6 

Policy Implications and Future Directions for Research 



Community colleges play an important role in serving the postsecondary education 
needs of almost half of all students enrolled in institutions of higher learning.' Given their 
relatively low cost and fee structures, community colleges are particularly important for 
low-income students. Yet, a preponderance of evidence suggests that community college 
students are not achieving success — more community college students drop out or “stop 
out” (that is, stop going to school for a semester or more) and very few obtain a degree 
within six years of matriculation.^ MDRC’s Opening Doors project sought to evaluate four 
potential strategies geared to increase academic achievement among diverse groups of 
community college students. 

This report presents the results from a rigorous study of an incentive scholarship pro- 
gram with counseling for low-income parents in two New Orleans-area community colleges 
that operated from 2004 to 2005 as part of the multisite Opening Doors demonstration. The 
findings are as follows: 

• Opening Doors encouraged program group students to register at greater 
rates during both the first and second semesters, when the program was in 
place. The findings suggest that program group students were more likely to 
register full time, earn more credits, and meet the grade point average (GPA) 
benchmark set by the program. 

• The program had positive effects on academic outcomes as late as the fourth 
semester after random assignment — two full semesters after most students 
exhausted their program eligibility. 

• Over several semesters, program group students experienced about a third of 
a semester gain. 

• Program group students exhibited greater levels of engagement in the pursuit 
of personal goals, self-esteem, and overall sense of self compared with con- 
trol group students. 



*U.S. Department of Education (2003b). 
^U.S. Department of Education (2003b). 
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This chapter explores the possible reasons why the enhanced performance scholarship 
program may have worked in Louisiana and presents important questions for which more re- 
search is needed prior to advocating for this strategy to be expanded. 



“Paying for Persistence”: Why Did it Work in Louisiana? 

The findings discussed in this report may be a direct result of the multidimensional 
model used in Louisiana, which addressed four concerns simultaneously. First, like typical fi- 
nancial aid programs, the Louisiana model acknowledged that college costs matter, especially 
for low-income students. The program targeted those with income beneath 200 percent of the 
federal poverty level and provided funds that were paid in addition to other financial aid for 
which the students had already qualified. 

Second, the Louisiana model targeted nontraditional students, specifically single par- 
ents. Most financial aid programs, including federal programs, are designed specifically for tra- 
ditional students — students who graduated from high school the summer prior to starting col- 
lege, who continue to be dependent upon their parents for support, and who are not parents 
themselves. The Louisiana sample was older (with an average age of 25) and all sample mem- 
bers were parents. Several studies have suggested that older students may benefit more from 
financial aid than younger students because older students — who may not have parents or oth- 
er family members willing to contribute to their education — may have to borrow funds to at- 
tend college and because they may have dependents (and therefore cannot forgo earnings while 
in school).^ For example, one study found that older students, especially women, are more sen- 
sitive to price changes in college tuition than are younger students (and males).'* In addition, 
another study found that the introduction of a $1,000 child care allowance into the Pell Grant 
formula increased maternal enrollment and attainment.^ In fact, studies that have found more 
moderate effects for scholarships have mostly been conducted on populations of traditional col- 
lege student age (18 to 22).*’ 

Third, the incentive scholarship was distributed directly to students, which may have al- 
lowed them to address the financial needs that most prevented their attendance and success. It 
was also paid in intervals closely associated with academic success: upon enrollment, after mid- 
term exams, and upon course completion. While federal and other financial aid provide some 
incentives for students to make progress toward a degree, the incentives are “sticks” rather than 
“carrots” (that is, students are motivated by the prospect of becoming ineligible in future years 



^Long (2007). 

'*Seftor and Turner (2002). 

^Simmons and Turner (2004). 

*’For example, see Angrist, Lang, and Oreopoulos (2006). 




for poor performance). Pell Grants and other financial aid also go directly to the college or uni- 
versity to cover tuition and fees and may seem less “real” to students than a check they can cash.^ 

Finally, the amount of need-based aid provided in Louisiana is limited. For example, 
Table 6.1 shows grant aid awards for undergraduates in Louisiana, the five lowest ranked states 
in terms of total aid, and the five highest ranked states. The table shows that out of the 50 states 
(in addition to Washington, DC, and Puerto Rico) surveyed by the National Association of State 
Student Grant and Aid Programs (NASSGAP), Louisiana ranked nineteenth in terms of total 
undergraduate aid and fifteenth in terms of grant dollars per population. However, the second 
panel shows that of the $1 16.5 million awarded to undergraduates, close to 99 percent was merit- 
based, with no awards based solely on need. The table also shows that Louisiana had a 77.9 per- 
cent decrease in need-based grant aid in the 10-year period preceding the 2005-2006 academic 
year.* The second and third columns show figures for the average of the five lowest ranked 
states and the five highest ranked states. The small amount of need-based aid in Louisiana and 
the relative importance of merit-based aid suggests that the students in Louisiana may have been 
particularly receptive to a scholarship that is both need-based and predicated on future academic 
achievement. In other words, it could be that any need-based aid program implemented in Loui- 
siana could have generated findings similar to those in Opening Doors. 

The Louisiana results are impressive given how difficult it is to increase persistence 
among community college students. In addition, the results are compelling enough to have gar- 
nered the authority to conduct a demonstration study (H.R. 4137, Section 405) in the Higher 
Education Opportunity Act (P.L. 110-315), which was signed into law by President George 
Bush on August 14, 2008. While the study was intended to answer fairly simple questions 
(“Will a performance-based scholarship affect persistence? If so, by how much?”), it has generat- 
ed a host of additional policy-relevant questions. Because Louisiana is just one test, MDRC has 
made it a priority to replicate the program (and variations of it) to build more evidence on the 
potential of performance-based scholarships to help at-risk students attending community col- 
leges and nonselective universities in other states. The goal of the Performance-Based Scholar- 
ship (PBS) demonstration — MDRC’s replication demonstration — is to evaluate whether 

^The best explanation for this laek of equivalenee between an aeeounting entry that reduees debt and a di- 
reet payment is “mental aeeounting” (Thaler, 1999). Mental aeeounting is the observed tendeney of people to 
treat finaneial transaetions and sourees of ineome in a pieeemeal way rather than in terms of their impaets on 
overall finaneial position. Thus, earned money earries a different propensity to eonsume than does “found” 
money; gamblers treat won “house money” differently from eash brought from home. A eheek handed to a 
student, with no strings attaehed, may therefore be a more powerful ineentive than a similar reduetion in the 
student’s balanee owed. 

*The state had a eorresponding pereentage inerease of 1,275.8 pereent in non-need-based grant aid in the 
same period. The only other states to experienee a deerease in need-based aid over the same period were Geor- 
gia (-70.6 pereent), Hawaii (-17.8 pereent). North Dakota (-21.4 pereent). South Dakota (-100 pereent), and 
Wyoming (-25.6 pereent). 
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The Opening Doors Demonstration 
Table 6.1 

Undergraduate State Grant Aid Awards, 2005-2006 



Delgado Community College and Louisiana Technical College Report 



Grant Aid 


Louisiana 


Average of 5 
Lowest-Ranked 
States" 


Average of 5 
Highest-Ranked 
States'’ 


Total undergraduate aid awarded (in millions) 


$116.4 


$1.1 


$585.9 


State rank by total undergraduate aid amount" 


19 


- 


- 


State rank by total grant dollars/population 


15 


- 


- 


Total undergraduate expenditures (in millions) 


$116.5 


$14.8 


$612.3 


Aid based only on need (%)'* 


- 


64.2 


53.5 


Aid based only on merit (%)" 


98.7 


NA 


44.3 


Aid based on need and merit (%f 


1.2 


4.0 


33.5 


Special purpose awards (%)® 


0.1 


59.0 


10.5 


Uncategorized (%)'’ 


NA 


67.4 


NA 


Total need-based aid awarded (in millions) 


$1.5 


1.1 


$431.5 


1 0-year change in need-based grant aid awarded (%) 


-77.9 


-5.1 


95.1 


Primary need-based grant program' 


Maximum award 


$2,000 


$2,260 


$3,929 


Minimum award 


$200 


$240 


$295 


Number of recipients 


4,033 


1,556 


159,710 


Expenditures per recipient 


$360 


$716 


$2,074 



SOURCES: Data derived from Tables 1, 4, 7, 8, 10 and 1 1 of the National Association of State Student Grant 
and Aid Programs (NASSGAP), 3 Z'* Annual Survey of State Sponsored Student Financial Aid, 2005-2006 
Academic Year (2007). 

NOTES: Louisiana is provided for reference, as it was the host state for the performance-based scholarship 
evaluation under the Opening Doors Demonstration. While Louisiana had a 77.9 percentage decrease in need- 
based grant aid in the 10-year period proceeding the 2005-2006 academic year, the state had a corresponding 
percentage increase of 1,275.8 percent in non-need-based grant aid in the same period. The primary need-based 
program was designated by each individual state in NASSGAP's annual survey. 

NA = not available. 

“The 5 lowest-ranked states in total undergraduate aid awarded are, in rank order, Arizona, North Dakota, 
Alaska, Elawaii, and Wyoming. 

'“The 5 highest-ranked states in total undergraduate aid awarded are, in rank order. New York, California, 
Georgia, Florida, and Pennsylvania. 

“Rank amounts include all 50 of the United States; Washington, DC; and Puerto Rico. 

“•Arizona, California, Hawaii, and Louisiana do not have grant programs based only on need. 

“California and Pennsylvania do not have grant programs based only on merit. 

*^Of the lowest ranked states, only Arizona has a grant program based on need and merit. Georgia and New 
Y ork do not have grant programs based on need and merit. In contrast, all of California's expenditures are based 
on need and merit. 

®" Special purpose rewards" refers to grant programs with specific objectives usually targeted to specific 
groups of students. For example, a workforce development program in which recipients train for particular 
occupations would be represented in this category. 

'’Of the lowest-ranked states, only Alaska has uncategorized awards. 

‘Louisiana's primary need-based grant program is the LA Leveraging Education Assistance Partnership 
(LEAP). The primary need-based grant program was designated by each individual state. 
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such scholarships would be as successful in different geographical locations, for different target 
groups, with different amounts of monies over varying durations. 



Important Unanswered Questions 

Several aspects of the Louisiana design may be critical for its success, but without 
studies of the variation in elements, it is not possible to determine the relative importance of 
each individual component. For example, the financial aid context (such as the relatively 
low amount of need-based aid offered to students in Louisiana) may play a very large role 
in generating the findings or, alternatively, the maturity of the students may be primarily 
responsible for the positive findings.'* Further, it is not clear from the present study whether 
the positive findings would have continued over time or reversed given the impact of Hurri- 
cane Katrina and its aftermath on the state and on surrounding regions. 

As a result, the study’s findings raise six important questions for future research: 

1 . What is the mechanism driving the students’ behavior? 

2. Are the findings limited to nontraditional students? 

3. How important is the counseling component? 

4. What amount of scholarship matters? Over what period of time? 

5. Does offering an incentive scholarship during the summer improve persis- 
tence and decrease the amount of time it takes, on average, to earn a degree? 

6. Will performance-based scholarships help students who require remediation? 

Causal Mechanism 

Why did the intervention in Louisiana generate improvements in enrollment, semester- 
to-semester persistence, and credit acquisition? Did the intervention’s provision of counseling 
make the difference by making students feel supported by the institution and thus more en- 
gaged? Are the results driven by the additional monetary contribution to the student’s house- 
holds? If so, did it matter because the additional money allowed them to reduce work hours and 
devote more time to studying? Did it enable them to purchase more reliable child care or child 
care they could not afford previously? Answers to these questions are key to understanding why 
the intervention may work to alter student behavior. 



^However, early findings from an evaluation of an enhaneed eounseling program for a similar target group 
of low-ineome parents (Serivener and An, 2007; Serivener and Pih, 2007) found very small impaets while the 
program was in operation and no impaets after the program ended. 
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Given that course grades presumably reflect a better understanding of the course con- 
tent and that enrolling for more credits requires more time for studies, it is reasonable to assume 
that a key mechanism by which the enhanced incentive scholarships improve student outcomes 
is increased effort while in school. Future research should examine whether the incentive built 
into the enhanced scholarships induces students to exert more effort on their studies. In other 
words, future work should be able to analyze whether students who are randomly assigned to 
receive an enhanced scholarship are spending more time in activities that would improve their 
academic outcomes (like attending class and studying) and spending less time in activities that 
would detract from their academic success (for instance, working or socializing with friends). 

Nontraditional Students 

The primary need-based financial aid program for college students in the United States 
is the Pell Grant program. This federal program served about 5.1 million students in the 2003- 
2004 academic year, offering students an average annual grant of $2,466.'° To be eligible for a 
Pell Grant, a student must be an undergraduate, show financial need, not be incarcerated in a 
federal or state penal institution, and not have yet earned a baccalaureate degree or its equiva- 
lent." Pell Grant receipt is typically lower for nontraditional students than for students attending 
college directly out of high school, for several reasons. 

First, nontraditional students are more likely than traditional students to attend college 
part time, work full time while enrolled in college, be financially independent, be displaced 
workers or unemployed, or be immigrants.'^ Each of these characteristics may be associated 
with lower Pell Grant receipt. Since financial aid is based on the cost of attendance, part-time 
attendance results in lower financial aid. Students who enroll for one credit-bearing class or 
several non-credit-bearing classes may not qualify for any financial assistance. Working full 
time or being financially independent results in unfavorable treatment in financial needs calcu- 
lations, which generally favor students with only limited amounts of income (typical of students 
just leaving high school).'^ Small changes in income can result in large changes in the amount a 
student is expected to contribute toward his or her education and much lower amounts of finan- 
cial aid. Students who are displaced workers or who are unemployed may be treated particularly 
unfavorably since financial needs analyses assume that the prior year’s income will be available 
to support a student for the current academic year. Hence, it may be the case that an enhanced 
scholarship, such as the one studied in this report, may generate large impacts among nontradi- 



'"National Association of Student Financial Aid Administrators (2005), p. 3. 

"U.S. Department of Education (2003a). Graduate students are also eligible for a Pell Grant if they are 
enrolled in an eligible post-baccalaureate teaching credential program. 

"'Choy (2002); Kazis et al. (2007). 

'^Long (2007). 



92 




tional students because other sources of financial aid are minimal, whereas students who are of 
traditional college age may have more of their unmet need covered by financial aid and there- 
fore may not have as strong an incentive to perform well academically. 

Additional Counseling 

Even though academic guidance and counseling may arguably be the most important 
student service, most students receive minimal help. Nationally, the average community college 
employs one adviser for approximately every 1,000 students. While advisers at different col- 
leges may deliver their services differently and cover different topics during advising sessions, 
the necessity of working with many students tends to drive them toward a traditional problem- 
solving approach in which a student presents an issue and the adviser offers a quick response. 

The National Academic Advising Association urges community colleges and four-year 
colleges and universities to provide sufficient staffing, so that students and advisers can have 
ongoing, interactive relationships, and to adopt a developmental approach whereby advisers 
help students clarify personal goals and objectives, rather than simply approving their choice of 
courses.'^ Research suggests that this type of advising is even more important for students who 
are also low-wage workers, since they may require more help than their younger counterparts in 
navigating their way to a credential.'*’ 

While the Louisiana model did include a counseling component, counselors generally 
had little formal training in academic advising or social work. Perhaps as a result, they did not 
usually discuss students’ selection of courses or choice of majors, nor did they delve into stu- 
dents’ personal lives. The implementation research in this report suggests that counselors de- 
voted most of their time to making sure students were enrolled at least half time and earning 
passing grades. That is, they largely provided a monitoring function. In contrast, in the Opening 
Doors Ohio sites mentioned in Chapter 1, students were assigned to one of a team of advisers, 
with whom they were expected to meet frequently to discuss academic progress and issues that 
might affect their schooling. Students also sometimes met with other advisers on the team and a 
designated contact in the financial aid office. Early findings from that intervention — which 
served similar types of students to those in the Louisiana program — suggest that such advising 
resulted in small changes in persistence but no differences in total credits earned.'^ While there 
is reason to believe that the monitoring function in this study was essential to the success of the 



'''Grubb (2001). 

'^Gordon, Habley, and Associates (2000). 

“■Kazis et al. (2007). 

'^Scrivener and An (2007); Scrivener and Pih (2007). 
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intervention, based on the Ohio results it may be reasonable to conclude that academic advising 
did not play a major role in the findings. 

Yet, the survey findings and interviews with administrators in Chapter 3 suggest that 
the monitoring and modest counseling that did occur in Louisiana seemed to greatly affect stu- 
dents (see the discussion of Table 3.4). Other research is also suggestive that counseling in con- 
junction with an incentive scholarship matters more than offering counseling alone.'* Clearly, 
more research is needed to understand the importance of counseling in relation to an incentive 
scholarship program. 

Amount and Duration of Scholarship 

The average tuition and fees at community colleges in 2003-2004 was $1,905, while the 
total cost of attendance was $10,981. '^In comparison, the average total cost of attending Delga- 
do Community College was $11,789 in school year 2003-2004, of which $1,656 (on average) 
was covered through financial aid, leaving an unmet need of $10, 133.^° In Louisiana, the $2,000 
scholarship reduced unmet need by slightly less than 20 percent. The two-semester scholarship 
conceivably could have assisted students in acquiring at least 12 college credits. While impor- 
tant, some studies suggest that acquisition of less than 20 credits within a one-year time frame is 
associated with a failure to attain a degree.^' 

Would a greater decrease in unmet need generate larger impacts on academic out- 
comes? Would a smaller scholarship generate similar gains, at lower implementation costs? Do 
impacts vary by the number of semesters that scholarships are offered? Or is the combination of 
counseling and scholarships for longer durations important? Should performance-based scholar- 
ships be calibrated for full-time and part-time students in such a way as to encourage more full- 
time attendance? These are important questions to answer, for both policymakers and private 



'*Angrist, Lang, and Oreopoulos (2006). 

'^College Board (2003), Table 3. Average total expenses inelude books and supplies, transportation, other 
expenses, and room and board costs for commuter students, representing the average estimated living expenses 
for students living off campus but not with parents. The equivalent figures for 2007-2008 are $2,361 in tuition 
and fees and $13,126 in total expenses, reflecting the increase in college attendance costs (College Board, 
2007b, Table 2). 

^"MDRC calculations using data from the Integrated Postsecondary Data System (IPEDS). Estimate 
represents an approximation since IPEDS data do not provide an average financial aid amount, but rather the 
average amounts of federal grant aid, state/local grant aid, institutional grant aid, and student loan aid. 

^’Adehnan (2006), p. 88, finds a weak negative relationship between the acquisition of 20 or less credits 
during the first year and the likelihood of earning a bachelor’s degree among students who initially matriculate 
at community colleges. Adehnan (1999) finds that students who attend four-year colleges and who earn fewer 
than 20 credits in their first calendar year of postsecondary experience have lower chances of completing a 
bachelor’s degree program. 
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scholarship foundations, who do not know what amount of aid will make a difference for the 
students they seek to help. 

Rewarding Summer Attendance 

Chapter 4 noted that the data display an interesting pattern suggesting that summer at- 
tendance may matter. As discussed in Chapter 4 and shown in Appendix Table C.l, while the 
patterns are suggestive that paying for persistence may go further if pay is granted for summer 
attendance, they are not definitive. The analyses cannot disentangle the effect of the incentive 
scholarship (with counseling) from summer attendance. 

The cleanest way to disentangle what happened in Louisiana is to replicate the design 
using the same target group in a similar environmental context (that is, in a place that pro- 
vides a similar level of financial aid outside of the intervention) and offer a summer stipend to 
a random sample of both program group students and control group students. Hence, the 
study would be a four-group design. Comparison of the control group eligible for a summer 
scholarship with the control group ineligible to receive a scholarship at any time would give 
the pure effect of offering a scholarship for summer study. Comparison of the program group 
eligible for the summer scholarship with the program group ineligible for a summer scholar- 
ship would provide the effect of adding a summer component to an incentive scholarship pro- 
gram. Both impacts have policy relevance, and such a design would provide significant 
knowledge to educators and policymakers. 

Students Requiring Remediation 

As a result of open admission policies, community colleges serve a large number of 
students who are academically underprepared for college-level work. Such students typically 
have to take courses in basic skills in math, reading, and writing before being able to register in 
certain college-level courses. This process of remediation is expensive for students since they 
must pay for the remedial class credits that do not apply toward a degree or certificate, and, if 
such students are receiving federal financial aid, they mn the risk of exhausting their aid eligibil- 
ity before they have obtained a degree or certificate.^^ Given the vast availability of remedial 
courses and the high prevalence of students requiring remediation in at least one subject, stu- 
dents are more likely to remain in a remediation sequence for a non-optimal period of time.^^ A 
performance-based scholarship may be able to help some portion of these students exit out of 
the remediation sequence by providing financial support that could supplant federal aid (allow- 



^^Levin and Calcagno (2007). 
^^Deil-Amen and Rosenbaum (2002). 
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ing that source of aid to be used for credit classes that count toward a degree) and an incentive 
to achieve academically. 



Looking Ahead 

While the Opening Doors Louisiana study was conducted utilizing random assignment, 
it is only one study in one particular place. In designing the Performance-Based Scholarship 
(PBS) demonstration, MDRC is seeking to fill gaps in federal and state financial aid, reach 
those students most in danger of falling through the cracks of higher education for financial rea- 
sons, and investigate the case for proposing performance-based scholarships as an alternative to 
the growing provision of merit aid in lieu of need-based aid. 

With a consortium of public and private funders, the PBS demonstration is anticipated 
to eventually include 6 to 8 states and 10 to 15 postsecondary institutions. As of this writing, 
the demonstration is operating in six sites: Borough of Manhattan and Hostos Community Col- 
leges in New York City; Lorain County, Owens, and Sinclair Community Colleges in Ohio; 
and the University of New Mexico in Albuquerque. In addition, the demonstration is operating 
through a program in California that works with students throughout the state. Table 6.2 pro- 
vides more background on the intervention implemented in each state. The table shows that the 
program in New York City will serve older students (22 to 35 years of age) who require at 
least one developmental-level course at the point of random assignment. The Ohio program 
serves low-income parents, a population similar to Louisiana’s, with differences in the scholar- 
ship amount (calibrated for part-time and full-time credits earned), and the intervals of dis- 
bursement. The program at the University of New Mexico serves a traditional college-going 
population and provides a $1,000 scholarship for each of four consecutive semesters if bench- 
marks are met. The California program will vary both the duration of scholarships (a range of 
options from one semester to four semesters) and the amount per period (either $500 or 
$1,000), while allowing the scholarships to be used at any institution. 
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The Opening Doors Demonstration 
Table 6.2 



Characteristics of Performance-Based Scholarship Demonstration Sites 
Delgado Community College and Louisiana Technical College Report 



Characteristic 


New York City 


Ohio 


New Mexico 


California 


Sample selection criteria 










Age (years) 


22-35 


18 or older 


16-26 


16-26 








(primarily 17-19) 


(primarily 17-19) 


Must be a parent? 


No 


Yes 


No 


No 


Collegiate level 


All 


All 


First year 


First year 


Remediation-level requirement 


At least one 


None 


None 


None 




course 








Intervention 










Scholarship amount per 


$1,300 


$900, full 


$1,000 


$500 or $1,000 


semester 




time; 










$450, part 










time 






Scholarship duration 


2 semesters 


2 semesters 


4 semesters 


1 to 4 semesters 


Scholarship distribution 








To be determined 


At enrollment 


15% 


— 


25% 


— 


At midterm 


35% 


— 


25% 


— 


At term end 


50% 


100% 


50% 


- 


Advising at scholarship 
distribution? 


No 


No 


Yes 


No 


Scholarship limited to 
institution (nontransferable)? 


Yes 


Yes 


Yes 


No 


Experimental design 










Number of schools 


2 


3 


1 


Many 


Projected total number of 
students in program group and 
control group 


1,500 


2,450 


1,000 


4,600 


Number of program groups 


1 


1 


1 


6 
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Appendix A 

Supplementary Table for Chapter 2 




The Opening Doors Demonstration 
Appendix Table A.l 

Selected Characteristics of Research Sample at Baseline 



Delgado Community College and Louisiana Technical College Report 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Gender (%) 


Male 


7.6 


8.9 


6.2 


Female 


92.4 


91.1 


93.8 


Average age (years) 


25.3 


25.2 


25.3 


Age(%) 


17-18 


4.1 


3.8 


4.5 


19-20 


13.8 


14.7 


13.0 


21-34 


82.0 


81.6 


82.5 


Marital status (%) 


Married, living with spouse 


8.2 


8.8 


7.5 


Married, not living with spouse 


10.8 


11.0 


10.5 


Unmarried, living with partner 


6.4 


5.2 


7.5 


Unmarried, not living with partner 


lA.l 


74.9 


74.4 


Race/ethnicity“ (%) 


Hispanic/Latino 


2.6 


3.0 


2.2 


Black, non-Hispanic 


84.9 


86.8 


83.0 * 


White, non-Hispanic 


10.5 


8.7 


12.2 * 


Asian or Pacific Islander 


0.4 


0.2 


0.6 


Multiracial 


0.7 


0.8 


0.6 


Other 


0.4 


0.0 


0.8 ** 


Number of children (%) 


1 


50.9 


51.7 


50.1 


2 


212 


29.8 


24.7 * 


3 or more 


21.9 


18.5 


25.2 *** 


Average age of youngest child (years) 


3.1 


3.1 


3.2 


Household receiving any government benefits (%) 


70.9 


72.4 


69.5 


Unemployment/Dislocated Worker benefits 


4.4 


5.1 


3.8 


Supplemental Security Income (SSI) or disability benefits 


13.2 


14.2 


12.2 


Cash assistance or welfare (TANF) 


10.3 


10.5 


10.2 


Food stamps 


61.8 


61.7 


62.0 


Public or Section 8 housing 


18.0 


15.4 


20.6 ** 


Financially dependent on parents (%) 


17.2 


17.6 


16.8 


Among those ever employed (%) 


97.7 


98.0 


97.5 


Average hourly wage at current or last job (S) 


7.27 


7.28 


7.26 



(continued) 
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Appendix Table A.l (continued) 



Characteristic 


Full 

Sample 


Program 

Group 


Control 

Group 


Among those employed at random assignment (%) 


51.6 


51.0 


52.2 


Average hourly wage at random assignment ($) 


7.34 


7.30 


7.38 


Diplomas/degrees earned'’ (%) 


High school diploma 


77.9 


79.2 


76.6 


General Educational Development (GED) certificate 


18.6 


17.0 


20.1 


Occupational/technical certificate 


9.8 


10.3 


9.4 


Date of high school graduation/GED receipt (%) 


During the past year 


10.7 


10.8 


10.5 


Between 1 and 5 years ago 


31.6 


32.3 


30.9 


Between 5 and 10 years ago 


33.2 


33.2 


33.3 


More than 10 years ago 


24.5 


23.6 


25.3 


Main reason for enrolling in college'’ (%) 


To complete a certificate program 


13.5 


12.7 


14.4 


To obtain an associate's degree 


56.3 


57.5 


55.1 


To transfer to a 4-year college/university 


15.5 


15.9 


15.2 


To obtain/update job skills 


13.3 


12.3 


14.4 


Other 


6.1 


6.0 


6.1 


Completed any college courses/credits (%) 


33.7 


32.7 


34.8 


First person in family to attend college (%) 


42.6 


42.8 


42.4 


Working personal computer in home (%) 


50.3 


49.7 


50.9 


Own or have access to a working car (%) 


68.9 


68.2 


69.6 


Respondent born in the United States'’ (%) 


97.7 


97.6 


97.7 


Respondent or 1 or more parents bom outside the United States'’ (%) 


4.6 


4.8 


4.5 


Sample size 


1,019 


505 


514 



SOURCE: MDRC calculations using Baseline Information Form (BIF) data. 

NOTES: Calculations for this table used all available data for the 1,019 sample members who completed a BIF. 

A two-tailed t-test was applied to differences between the research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and site. 

Italic type indicates nonexperimental data. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

^Respondents who indicated that they are Hispanic and who also chose a race are included only in the 
Hispanic/Latino category. 

'’Distributions may not add to 100 percent because categories are not mutually exclusive. 

‘’"United States" includes Puerto Rico. 
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Appendix B 

Survey Response Bias Analysis 




This appendix discusses the final survey sample used throughout the remainder of this report. 



Final Survey Samples and Survey Response Rate 

There were 1,019 sample members in the Opening Doors study at the Louisiana sites — 
Delgado Community College and Louisiana Technical College. Sample members were sur- 
veyed using one of two versions of the Opening Doors 12-month survey, which asked respon- 
dents about a wide range of topics, including their educational experiences, social relationships, 
and health. Between December 7, 2004, and September 1, 2005, the original version of the 12- 
month survey was administered. Between March 6, 2006, and January 8, 2007, following Hur- 
ricane Katrina, a revised version of the survey was administered, which included an initial mod- 
ule asking questions specifically related to the hurricane. 

Six sample members were excluded from the original version of the 12-month survey 
because they were ineligible (two individuals), incapacitated (three individuals), or incarcerated 
(one individual), leaving 1,013 fielded sample members. Of the 1,013 surveys fielded, 801 
sample members responded, for an overall response rate of 79.1 percent. The original 12-month 
survey was fielded to 614 sample members, and 492 (80.1 percent) responded (Group A in Fig- 
ure 2.3). The remaining 399 sample members were fielded using the 12-month survey with the 
hurricane module, and 309 members (77.4 percent) responded (Group B in Figure 2.3). The 
final research sample consists of the 801 sample members (Group A plus Group B in Figure 
2.3) who responded to either the original 12-month survey or the 12-month survey with the hur- 
ricane module. 



Assessment of Selective Survey Response 

Background Characteristics 

Appendix Table B.l compares the background characteristics of the 12-month survey 
respondents with the characteristics of the nonrespondent sample members. Characteristic 
means for the full sample are presented in the first data column, followed by means for the sur- 
vey respondents and for the nonrespondent sample members, with asterisks indicating whether 
the differences in the means of these two groups are statistically significant (meaning that the 
differences are unlikely to have arisen by chance). 

The table indicates that 93.6 percent of respondents were women compared with 88.4 
percent of nonrespondents, and 20.1 percent of respondents reported that they were married 
compared with 14.5 percent of nonrespondents. Both differences are statistically significant. 
Younger individuals were also somewhat more likely to respond; 14.8 percent of respondents 
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The Opening Doors Demonstration 
Appendix Table B.l 

Characteristics of Twelve-Month Survey Respondents and Nonrespondents 



Delgado Community College and Louisiana Technical College Report 



Characteristic (%) 


Full 

Sample 


Respondents 


Nonrespondents 


Gender 








Male 


7.5 


6.4 


11.6 ** 


Female 


92.5 


93.6 


88.4 ** 


Age (years) 








17-18 


4.1 


4.5 


2.9 


19-20 


13.7 


14.8 


9.6 * 


21-34 


82.1 


80.7 


87.5 ** 


Marital status 








Married 


19.0 


20.1 


14.5 * 


Unmarried 


81.0 


79.9 


85.5 * 


Race/ethnicity* 








Hispanic/Latino 


2.7 


2.9 


1.6 


Black, non-Hispanic 


84.9 


84.8 


85.4 


White, non-Hispanic 


10.5 


10.1 


12.1 


Asian or Pacific Islander 


0.4 


0.4 


0.5 


Other'’ 


0.4 


0.4 


0.4 


One child in household 


1.0 


1.0 


1.0 


Household receiving any government benefits'’ 


70.8 


71.7 


67.0 


Financially dependent on parents 


17.3 


17.6 


16.1 


Ever employed 


97.7 


97.9 


97.0 


Currently employed 


51.7 


49.8 


59.1 ** 


Diplomas/degrees earned"^ 








High school diploma 


77.8 


79.1 


72.6 ** 


General Educational Development (GED) certificate 


18.7 


17.3 


24.1 ** 


Occupational/technical certificate 


9.9 


10.4 


7.9 


Date of high school graduation/GED receipt 








During the past year 


10.6 


10.7 


10.5 


Between 1 and 5 years ago 


31.7 


32.1 


30.0 


More than 5 years ago 


57.7 


57.2 


59.5 



(continued) 
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Appendix Table B.l (continued) 



Characteristic (%) 


Full 

Sample 


Respondents 


Nonrespondents 


Main reason for enrolling in college‘s 
To complete a certificate program 


13.6 


14.2 


11.5 


To obtain an associate's degree 


56.4 


55.6 


59.0 


To transfer to a 4-year college/university 


15.6 


15.9 


14.6 


To obtain/update job skills 


13.2 


13.9 


10.8 


Other 


6.0 


5.5 


7.8 


First person in family to attend college 


42.5 


42.6 


42.0 


Working personal computer in home 


50.3 


50.6 


49.2 


Owns or has access to a working car 


68.9 


69.0 


68.6 


Language other than English spoken regularly in home 


7.0 


6.7 


8.1 


U.S. citizen 


99.0 


99.0 


99.0 


Respondent bom outside United States'* 


0.0 


0.0 


0.0 


Respondent or respondent's parent(s) 
bom outside United States'* 


4.7 


4.8 


4.0 


Sample size 


1,013 


801 


212 



SOURCE; MDRC calculations using Baseline Information Form (BIF) data. 

NOTES; A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as; *** = 1 percent; ** = 5 percent; * = 10 percent. 

To analyze whether baseline characteristics and research group status predicted survey response, an 
omnibus F-test was performed, which yielded a p-value of 0.01. This suggests there may be statistically 
significant differences in observable characteristics between survey respondents and nonrespondents. 

Estimates are adjusted by round of random assignment . 

Missing values are not included in individual variable distributions. 

Distributions may not add to 100 percent because of rounding. 

“Respondents who indicated that they are Hispanic and who also chose a race are included only in the 
Hispanic/Latino category. 

’’"Other" race/ethnicity includes American Indians/Alaskan Natives and those who marked more than 
one category. 

“Distributions may not add to 100 percent because categories are not mutually exclusive. 

“•"United States" includes Puerto Rico. 

“This category includes the Commonwealth of Independent States, which comprises Armenia, 
Azerbaijan, Belarus, Georgia, Kazakhstan, Kyrgyzstan, Republic of Moldova, Russia, Tajikistan, 
Turkmenistan (until August 2005), Ukraine, and Uzbekistan. Other regions include the Baltics, eastern and 
western Europe, North Africa, Sub-Saharan Africa, the Near East, and Oceania. Countries are grouped by 
region according to the U.S. Bureau of the Census, International Data Base. 

Hhe majority of respondents (over 80 percent) reported that both parents were bom in the same region. 
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were 19 to 20 years of age compared with 9.6 percent of nonrespondents, a difference that is 
statistically significant at the 10 percent level. These data also indicate some statistically signifi- 
cant differences in a few educational and employment characteristics at the point of random 
assignment. In particular, respondents were less likely to report being currently employed, more 
likely to report that they had earned a high school diploma, and less likely to report having a 
General Educational Development (GED) diploma. 

Appendix Table B.2 presents baseline characteristic means for all 12-month survey 
respondents, means for the program group respondents, and means for the control group 
respondents. The table also indicates which characteristics show statistically significant 
differences between the program and control groups, as described in the table notes. While the 
mean characteristics presented in Table B.l suggest that survey respondents were in several ways 
different from sample members who did not respond to the survey, these same characteristics do 
not differ between those respondents who were randomly assigned to participate in the program 
and those randomly assigned to the control group. Indeed, among respondents, program group and 
control group respondents show marginally statistically significant differences on only two 
race/ethnicity categories, “Asian or Pacific Islander” and “Other.” 
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The Opening Doors Demonstration 
Appendix Table B,2 

Selected Characteristics of Twelve-Month Survey Respondents, 

by Research Group 



Delgado Community College and Louisiana Technical College Report 



Characteristic (%) 


Full 

Sample 


Program 

Group 


Control 

Group 


Gender 


Male 


6.4 


7.5 


5.2 


Female 


93.6 


92.5 


94.8 


Age (years) 


17-18 


4.5 


3.7 


5.3 


19-20 


14.7 


15.9 


13.5 


21-34 


80.8 


80.3 


81.2 


Marital status 


Married 


20.2 


21.4 


19.0 


Unmarried 


79.8 


78.6 


81.0 


Race/ethnicity“ 


Hispanic/Latino 


3.0 


3.3 


2.6 


Black, non-Hispanic 


84.6 


86.3 


82.9 


White, non-Hispanic 


10.2 


8.9 


11.6 


Asian or Pacific Islander 


0.4 


0.0 


0.8 * 


Other'’ 


0.4 


0.0 


0.8 * 


One child in household 


1.0 


1.0 


1.0 


Household receiving any government benefits'’ 


71.7 


73.4 


70.0 


Financially dependent on parents 


17.6 


18.2 


17.1 


Ever employed 


97.9 


98.5 


97.3 


Currently employed 


49.6 


49.9 


49.2 


Diplomas/degrees earned'’ 


High school diploma 


78.9 


79.3 


78.4 


General Educational Development (GED) certificate 


17.5 


16.4 


18.6 


Occupational/technical certificate 


10.4 


9.9 


10.9 


Date of high school graduation/GED receipt 


During the past year 


10.7 


11.0 


10.5 


Between 1 and 5 years ago 


32.0 


33.5 


30.6 


More than 5 years ago 


57.3 


55.5 


59.0 



(continued) 
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Appendix Table B,2 (continued) 



Characteristic (%) 


Full 

Sample 


Program 

Group 


Control 

Group 


Main reason for enrolling in college‘s 
To complete a certificate program 


14.2 


13.7 


14.7 


To obtain an associate's degree 


55.6 


57.5 


53.8 


To transfer to a 4-year college/university 


16.0 


16.4 


15.6 


To obtain/update job skills 


13.8 


12.2 


15.4 


Other 


5.4 


5.0 


5.9 


First person in family to attend college 


42.8 


42.3 


43.3 


Working personal computer in home 


50.6 


49.6 


51.6 


Owns or has access to a working car 


68.9 


67.3 


70.5 


Language other than English spoken regularly in home 


6.8 


6.2 


7.5 


U.S. citizen 


99.0 


99.0 


99.0 


Respondent born outside United States'* 


0.0 


0.0 


0.0 


Respondent or respondent's parent(s) 
bom outside United States'* 


4.8 


4.9 


4.6 


Sample size 


801 


402 


399 



SOURCE: MDRC calculations using Baseline Infomation Form (BIF) data. 

NOTES: Calculations for this table used all available data for the 61 1 survey respondents who were 
randomly assigned from April 4 through August 17, 2005. 

A two-tailed t-test was applied to differences between the groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

To analyze whether baseline characteristics predicted treatment among this group of survey 
resopndents, an omnibus F-test was performed, which yielded a p-value of 0.95. This suggests there are no 
statistically significant differences in observable characteristics between program group and control group 
students who responded to the survey. 

Estimates are adjusted by college and research group. 

Missing values are not included in individual variable distributions. 

Distributions may not add to 1 00 percent because of rounding. 

“Respondents who indicated that they are Hispanic and who also chose a race are included only in the 
Hispanic/Eatino category. 

'’"Other" race/ethnicity includes American Indians/ Alaskan Natives and those who marked more than 
one category. 

‘’Distributions may not add to 100 percent because categories are not mutually exclusive. 

‘'"United States" includes Puerto Rico. 
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Appendix C 

Supplementary Tables for Chapter 4 




The Opening Doors Demonstration 
Appendix Table C.l 
Enrollment Outcomes, by Cohort 



Delgado Community College and Louisiana Technical College Report 







Relative Semester of Enrollment 




Cohort 


First 

Semester 


Second 

Semester 


Third 

Semester 


Fourth 

Semester 


Cohort 1 


Actual semester 


Spring 2004 


Summer 2004 


Fall 2004 


Spring 2005 


Program group (%) 


85.9 


52.3 


48.7 


36.9 


Control group (%) 


85.7 


28.2 


37.9 


31.0 


Difference 


0.2 


24 1 *** 


10.9 ** 


6.0 


Number of observations 


289 


289 


289 


289 


Cohort 2 


Actual semester 


Summer 2004 


Fall 2004 


Spring 2005 


Summer 2005 


Program group (%) 


61.1 


62.6 


50.1 


22.2 


Control group (%) 


51.2 


51.4 


37.3 


13.5 


Difference 


9.9 * 


11.2 * 


12.9 ** 


8.7 * 


Number of observations 


248 


248 


248 


248 


Cohort 3 


Actual semester 


Fall 2004 


Spring 2005 


Summer 2005 


Fall 2005 


Program group (%) 


90.8 


78.5 


25.2 


7.7 


Control group (%) 


86.8 


70.6 


26.3 


6.8 


Difference 


3.9 


7.9 * 


-1.1 


1.0 


Number of observations 


385 


385 


385 


385 


Cohort 4 


Actual semester 


Spring 2005 


Summer 2005 


Fall 2005 


Spring 2006 


Program group (%) 


91.0 


53.3 


4.2 


12.4 


Control group (%) 


76.4 


27.9 


4.0 


16.4 


Difference 


14.6 ** 


25.4 ** 


0.2 


-4.0 


Number of observations 


97 


97 


97 


97 



SOURCES; MDRC calculations from transcript data obtained from Delgado Community College and 
Louisiana Technical College-West Jefferson. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by campus. 
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The Opening Doors Demonstration 
Appendix Table C,2 
Scholarship Receipt: First Two Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome (%) 


Program 

Group 


Control 

Group 


Difference 


Standard 

Error 


Received any scholarship payment in: 
First semester 


76.1 


0.4 


"75 y *** 


2.5 


Second semester 


53.7 


0.4 


53.3 *** 


2.9 


Third semester 


9.8 


0.0 


p g *** 


1.8 


Fourth semester 


1.1 


0.0 


1.1 * 


0.6 


Fifth semester 


0.4 


0.0 


0.4 


0.4 


Sample size (n = 537) 


264 


273 







SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College 
transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance 
levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 
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The Opening Doors Demonstration 
Appendix Table C.3 

Cumulative Educational Outcomes, First Semester Through 
Seventh Semester: All Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program Control 
Group Group Difference 


Standard 

Error 


Semesters 1 and 2 tnre-Katrina) 












Registered for any courses (%) 


87.2 


82.1 


5.1 


** 


2.1 


Number of semesters registered 


1.5 


1.3 


0.2 


*** 


0.0 


Number of credits earned 


9.8 


7.4 


2.3 


*** 


0.5 


Regular credits 


7.1 


5.3 


1.8 


*** 


0.4 


Equated credits 


2.7 


2.2 


0.6 


*** 


0.2 


Withdrew from one or more courses (%) 


45.4 


47.6 


-2.3 




3.0 


Cumulative GPA^ (%) 












No GPA'’ 


18.8 


25.5 


-6.7 


*** 


2.5 


Earned a 2.0 GPA or greater 


55.0 


46.9 


8.1 


*** 


3.0 


Earned less than a 2.0 GPA 


26.2 


27.6 


-1.5 




2.8 


Semesters 1 throueh 7 












Registered for any courses (%) 


88.7 


83.8 


4.9 


** 


2.0 


Number of semesters registered 


2.4 


2.1 


0.3 


*** 


0.1 


Number of credits earned 


14.9 


12.0 


2.9 


*** 


0.9 


Regular credits 


11.5 


9.1 


2.5 


*** 


0.8 


Equated credits 


3.4 


2.9 


0.5 


* 


0.2 


Withdrew from one or more courses (%) 


62.6 


60.1 


2.4 




2.9 


Cumulative GPA^ (%) 












No GPA'’ 


17.4 


22.2 


-4.7 


** 


2.4 


Earned a 2.0 GPA or greater 


50.9 


44.0 


6.9 


** 


3.0 


Earned less than a 2.0 GPA 


31.7 


33.8 


-2.1 




2.9 


Sample size (n = 1,019) 


505 


514 









(continued) 
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Appendix Table C.3 (continued) 



SOURCES: MDRC calculations from Delgado Community College and Louisiana Technical College 
transcript data. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

^Cumulative GPA is based on credit-bearing courses taken from random assignment through the end 
of the third postprogram semester. Courses in which students did not receive a passing grade and 
subsequently repeated are not included in the cumulative GPA. 

'’The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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The Opening Doors Demonstration 
Appendix Table C.4 

Cumulative Enrollment Outcomes at Delgado Community College, 
First Semester Through Seventh Semester: All Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program Control 
Group Group Difference 


Standard 

Error 


Semesters 1 and 2 fnre-Katrinal 










Registered (%) 










At any school 


90.6 


87.1 


3.5 * 


2.1 


At Delgado Community College 


90.1 


87.1 


3.0 


2.1 


For any course at Delgado (transcript) 


89.7 


86.4 


3.3 


2.2 


At any 2 -year school 


90.4 


87.1 


3.3 


2.1 


At any 4-year school 


0.7 


0.5 


0.3 


0.5 


Number of semesters enrolled at any school 


1.5 


1.4 


Q 2 *** 


0.0 


Number of semesters enrolled at Delgado (transcript) 


1.5 


1.4 


Q 2 *** 


0.0 


Semesters 1 through 7 










Registered (%) 










At any school 


93.3 


90.5 


2.8 


1.8 


At Delgado Community College 


91.6 


89.1 


2.6 


2.0 


For any course at Delgado (transcript) 


91.1 


88.6 


2.6 


2.0 


At any 2 -year school 


92.4 


89.1 


3.3 * 


1.9 


At any 4-year school 


9.4 


11.2 


-1.8 


2.1 


Number of semesters enrolled at any school 


2.8 


2.6 


0.2 


0.1 


Number of semesters enrolled at Delgado (transcript) 


2.6 


2.4 


0.2 ** 


0.1 


Sample size (n = 817) 


406 


411 







SOURCES: MDRC calculations using data from the StudentTracker service of the National Student 
Clearinghouse and transcript data for Delgado Community College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

The Clearinghouse collects data from about 3,100 colleges that enroll 91 percent of U.S. college 
students (http://www.studentclearinghouse.org/about/pdfs/Clearinghouse_profile.pdf). Students have the 
right to opt out of having their information sent to the Clearinghouse. MDRC was able to find records in the 
Clearinghouse file for 93 percent of the students randomly assigned at Delgado Community College 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

Distributions may not add to 100 percent because of rounding. 
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The Opening Doors Demonstration 
Appendix Table C.5 

Cumulative Educational Outcomes at Delgado Community College, 
First Semester Through Seventh Semester: All Cohorts 



Delgado Community College and Louisiana Technical College Report 



Outcome 


Program Control 
Group Group Difference 


Standard 

Error 


Semesters 1 and 2 tnre-Katrina) 












Registered for any courses (%) 


89.7 


86.4 


3.3 




2.2 


Number of credits earned 


10.1 


8.0 


2.2 


*** 


0.5 


Regular credits 


6.9 


5.4 


1.5 


*** 


0.4 


Equated credits 


3.3 


2.6 


0.6 


*** 


0.2 


Withdrew from one or more courses (%) 


47.6 


53.5 


-5.9 


* 


3.4 


Cumulative GPA^ (%) 












No OPA” 


16.0 


21.4 


-5.4 


** 


2.6 


Earned a 2.0 GPA or greater 


60.8 


53.1 


7.8 


** 


3.4 


Earned less than a 2.0 GPA 


23.2 


25.5 


-2.4 




3.0 


Number of semesters enrolled at any school 


1.5 


1.4 


0.2 


*** 


0.0 


Number of semesters enrolled at Delgado (transcript) 


1.5 


1.4 


0.2 


*** 


0.0 


Semesters 1 throush 7 












Registered for any courses (%) 


91.1 


88.6 


2.6 




2.0 


Number of credits earned 


15.8 


13.2 


2.7 


*** 


1.0 


Regular credits 


11.7 


9.6 


2.1 


** 


0.9 


Equated credits 


4.1 


3.6 


0.5 


* 


0.3 


Withdrew from one or more courses (%) 


68.2 


68.1 


0.1 




3.2 


Cumulative GPA^ (%) 












No GPA” 


14.6 


17.3 


-2.7 




2.5 


Earned a 2.0 GPA or greater 


55.6 


49.7 


6.0 


* 


3.5 


Earned less than a 2.0 GPA 


29.8 


33.1 


-3.3 




3.3 


Number of semesters enrolled at any school 


2.8 


2.6 


0.2 




0.1 


Number of semesters enrolled at Delgado (transcript) 


2.6 


2.4 


0.2 


** 


0.1 


Sample size (n = 817) 


406 


411 









(continued) 
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Appendix Table C.5 (continued) 



SOURCES: MDRC calculations using data from the StudentTracker service of the National Student 
Clearinghouse and transcript data for Delgado Community College. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

The Clearinghouse collects data from about 3,100 colleges that enroll 91 percent of U.S. college 
students (http://www.studentclearinghouse.org/about/pdfs/Clearinghouse_profile.pdf). Students have the 
right to opt out of having their information sent to the Clearinghouse. MDRC was able to find records in 
the Clearinghouse file for 93 percent of the students randomly assigned at Delgado Community College. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels 
are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

Distributions may not add to 100 percent because of rounding. 

“Cumulative GPA is based on credit-bearing courses taken from random assignment through the end 
of the third postprogram semester. Courses in which students did not receive a passing grade and 
subsequently repeated are not included in the cumulative GPA. 

'’The "No GPA" category includes students who did not enroll and students who took only 
developmental courses, which are not included in GPA calculations. 
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Appendix D 

Description of Scaies Presented in Chapter 5 




The following multi-item scale measures, which are presented in Chapter 5, were 
created using data from the Opening Doors follow-up survey. Multi-item scales are useful for 
measuring complex consfructs, such as those outlined below, because such constructs cannot be 
easily assessed with a single-item measure. Most of these scale measures have been widely used 
in related literature, and footnotes are added to reference the original source from which scales 
were drawn or adapted. Three measures — “sense of self,” “friends value education,” and “po- 
litical engagemenf’ — were developed for the Opening Doors demonstration.' Cronbach’s Al- 
pha, a measure of how well the items included in the scale measure a common underlying con- 
stmct," is presented for each scale. 



Social and Psychological Outcomes 

Outlook and Identity 

Optimism^ (6-item scale, Cronbach’s Alpha - .53)** 

1 . In uncertain times, I usually expect the best. 

2. If something can go wrong for me, it will, (reverse code) 

3. lam always optimistic about my future. 

4. I hardly ever expect things to go my way. (reverse code) 

5. I rarely count on good things happening to me. (reverse code) 

6. Overall, I expect more good things to happen to me than bad. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Goal Engagement (3-item scale, Cronbach’s Alpha - .62) 

1 . I don’t think much about my long-term goals, (reverse code) 

2. I have many long-term goals that I will work to achieve. 

3. It is important for me to take time to plan out where I’m going in life. 



'Questions included in these new measures are similar to those used elsewhere. 

^Cronbach (1951). 

^Scheier, Carver, and Bridges (1994). 

"'Data on a subset of measures are available at both baseline and follow-up. All Cronbach’s Alphas shown 
above were calculated using follow-up data. 

^Drawn from a measure of “reactive responding”; see Taylor and Seeman (1999). 
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Response categories: 



Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Life Engagement*’ (6-item scale, Cronbach’s Alpha - .77) 

1 . There is not enough purpose in my life, (reverse code) 

2. I don’t care very much about the things I do. (reverse code) 

3. To me, the things I do are all worthwhile. 

4. I have lots of reasons for living. 

5. Most of what I do seems trivial and unimportant to me. (reverse code) 

6. I value my activities a lot. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Self-Esteem^ (4-item scale, Cronbach’s Alpha - .60) 

1 . lam able to do things as well as most other people. 

2. I feel that I’m a person of worth, or at least on an equal basis with others. 

3 . I feel that I have a number of good qualities. 

4. I take a positive attitude toward myself 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 



^Scheier et al. (2006). 

^Adapted from Rosenberg (1965). 
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Sense of Self (13-item scale, Cronbach’s Alpha - .86) 

1 . Y our goals in life are becoming clearer. 

2. People know they can count on you to “be there” for them. 

3 . You have a clear sense of your beliefs and values. 

4. There is at least one person who knows “the real you.” 

5. You have a good deal of freedom to explore things in life that interest you. 

6. You feel respected by others as an adult. 

7. There is at least one person with whom you can talk about anything. 

8. You feel that you are important, that you “matter,” to other people. 

9. You have a pretty good sense of the path you want to take in life and the 
steps to take to get there. 

10. You can envision the kind of person you’d like to become. 

11. You feel your life is filled with meaning, a sense of purpose. 

12. It is easy for you to make close friends. 

13. People often seek your advice and support. 

Response categories: Strongly disagree (1) 

Somewhat disagree (2) 

Somewhat agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Social Support and Civic Engagement 

General Social Support^ (8-item scale, Cronbach’s Alpha - .82) 

1 . There are people I know will help me if I need it. 

2. There is no one I feel comfortable talking about problems with, (reverse 
code) 

3. I am with a group of people who think the same way I do about things. 

4. If something went wrong, no one would help me. (reverse code) 

5. I have a tmstworthy person to turn to if I have problems. 

6. I do not think that other people respect what I do. (reverse code) 

7. There is no one who likes to do the things I do. (reverse code) 

8. There are people who value my skills and abilities. 



^Adapted from Cutrona and Russell (1987). 
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Response categories: 



Strongly disagree (1) 

Disagree (2) 

Agree (3) 

Strongly agree (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Friends Value Education (5-item scale, Cronbach’s Alpha - .93) 

Among your friends, how important is it to. . . 

1. Go to college? 

2. Get good grades? 

3. Complete a college degree or training program? 

4. Use a college degree or program certificate to get a better job? 

5. Pursue advanced study after college? 

Response categories: Not very (1) 

Somewhat (2) 

Quite a bit (3) 

Extremely (4) 

Responses were summed and averaged. Scores range from 1 to 4. 

Political Engagement (4-item summative scale, Cronbach’s Alpha - .42) 

1 . Are you registered to vote? 

2. Did/do you plan to vote in the 2004 presidential election? 

3. Since [date of random assignment], have you donated your time or money 
to a political campaign? 

4. Since [date of random assignment] have you attended a political speech, 
rally, or march? 

Each item has two response categories (l=Yes and 0=No). The four items 
are added together and divided by four. Response range is 0 to 1. 
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Health Outcomes 



Mental Health 

Stress^ (4-item scale, Cronbach’s Alpha - .72) 

In the last 30 days, how often. . . 

1 . Have you felt you were unable to control the important things in your life? 

2. Have you felt confident about your ability to handle your personal 
problems? (reverse code) 

3. Have you felt that things were going your way? (reverse code) 

4. Have you felt difficulties were piling up so high that you could not 
overcome them? 

Response categories: None of the time (1) 

A little of the time (2) 

Some of the time (3) 

Most of the time (4) 

All of the time (5) 

Responses were summed and averaged. Scores range from 1 to 5. 

Psychological Distress”* 16-item summative scale, Cronbach’s Alpha - .74) 

During the past 30 days, about how often. . . 

1 . Did you feel nervous? 

2. Did you feel hopeless? 

3 . Did you feel restless or fidgety? 

4. Did you feel so depressed that nothing could cheer you up? 

5. Did you feel that everything was an effort? 

6. Did you feel worthless? 

Response categories: None of the time (0) 

A little of the time (1) 

Some of the time (2) 

Most of the time (3) 

All of the time (4) 

Responses were summed. Scores range from 0 to 24, with a cut-off point of 
1 3 to determine nonspecific psychological distress. 



^Adapted from Cohen, Kamarck, and Mermelstein (1983); Cohen and Williamson (1988). 
'^Kessler et al. (2002). 
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Appendix E 

Supplementary Tables for Chapter 5 




The Opening Doors Demonstration 
Appendix Table E.l 

Differences in Social, Psychological, and Health Measures 
of Survey Respondents and Nonrespondents at Baseline 

Delgado Community College and Louisiana Technical College Report 



Outcome 


Sample 

Size 


Survey 

Nonrespondent 


Survey 

Respondent 


Difference 


Standard 

Error 


General social support^ (1-4) 


978 


3.14 


3.20 


0.06 


0.04 


Perceived stress'’ (1-5) 


995 


2.40 


2.39 


-0.01 


0.06 


K6 score for psychological 
distress'’ (0-24) 


971 


4.72 


4.99 


0.27 


0.33 


Indicator of high 
psychological distress'* (%) 


971 


5.2 


5.5 


0.3 


1.8 


Health status fair or poor (%) 


998 


1.3 


3.6 


2.3 * 


1.4 


Body mass index” (kg/m^) 


987 


27.59 


28.53 


0.94 * 


0.55 


Overweight or obese 
(BMI > 25)''(%) 


987 


61.5 


64.2 


2.7 


3.8 


Current Smoker (%) 


984 


17.6 


9.4 


-8 2 *** 


2.5 



SOURCE: MDRC calculations from the Opening Doors Baseline Survey. 

NOTES: This table shows baseline characteristics of the sample members broken down by whether or not they 
responded to the Opening Doors 12-month survey. 

Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“An 8-item scale about the presence of social support; response categories range from 1 = "strongly 
disagree" to 4 = "strongly agree." Items are averaged. 

’’A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very 
often." Items are averaged. 

“A 6-item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

‘'An indicator if the K6 Screening Scale measure of psychological distress exceeds 12. 

“BMI = weight in kilograms divided by height in meters squared. 

^1 = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight less than 18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and 
obese = 30 or greater. 
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The Opening Doors Demonstration 
Appendix Table E.2 

Comparison of Baseline Characteristics of Groups A and B 
Delgado Community College and Louisiana Technical College Report 



Outcome 


Sample 

Size 


Full 

Sample 


Group A 


Group B 


Difference 


General social support'* (1-4) 


769 


3.20 


111 


3.10 


Q Y] 


Perceived stress'* (1-5) 


783 


2.39 


2.32 


2.50 


-0.18 ** 


K 6 score for psychological 
distress" (0-24) 


766 


4.95 


4.61 


5.50 


- 0.88 ** 


Indicator of high psychological 
distress'* (%) 


766 


5.5 


3.8 


8.1 


-4.2 * 


Health status fair or poor (%) 


786 


3.6 


3.9 


3.0 


0.9 


Body mass index" (kg/m^) 


778 


28.49 


28.99 


27.71 


1.29 


Overweight or obese (BMI > 25)*^(%) 


778 


64.0 


66.3 


60.3 


6.0 


Current smoker (%) 


111 


9.4 


8.7 


10.5 


-1.7 



SOURCE: MDRC calculations from the Opening Doors Baseline Survey. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“An 8 -item scale about the presence of social support; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." Items are averaged. 

'’A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very 
often." Items are averaged. 

‘^A 6 -item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

‘■An indicator if the K 6 Screening Scale measure of psychological distress exceeds 12. 

“^BMI = weight in kilograms divided by height in meters squared. 

'1 = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight less than 18.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and obese 
= 30 or greater. 
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The Opening Doors Demonstration 
Appendix Table E.3 
Treatment Effects for Groups A and B 
Delgado Community College and Louisiana Technical College Report 





Group A 

(12-Month Survey) 


Group B 

(12-Month Survey) 


Outcome 


Impact 


Standard 

Error 


Impact 


Standard 

Error 


Outlook and identity 










Optimism (1-4) 


0.06 


0.04 


-0.02 


0.05 


Goal engagement (1-4) 


0 j j *** 


0.04 


0.06 


0.05 


Life engagement (1-4) 


0.07 * 


0.04 


0.02 


0.05 


Self-esteem (1-4) 


0.08 ** 


0.04 


0.02 


0.05 


Sense of self (1-4) 


0.05 * 


0.03 


0.06 


0.04 


Social suDDort and civic eneaeement 










General social support ““ (1-4) 


0.09 ** 


0.04 


0.11 ** 


0.05 


Friends supportive of college (1-4) 


-0.11 


0.08 


0.02 


0.10 


Volunteerism'’ (%) 


4.6 


3.9 


-0.3 


5.0 


Political engagement (0-1) 


0.02 


0.02 


0.03 


0.02 


Antisocial behavior in nast year 










Spent time in reform school or prison (%) 


0.5 


1.8 


-0.2 


2.5 


Close friend spent time in reform school or 
prison (%) 


1.1 


3.9 


6.6 


4.9 


Health and health behaviors 










Perceived stress‘d (1-4) 


-0.10 


0.07 


0.08 


0.10 


K6 score for psychological distress'* (0-24) 


-0.14 


0.38 


-0.05 


0.53 


Indicator of high psychological 
distress'^ (%) 


-1.3 


2.2 


0.9 


3.5 


Health status fair or poor (%) 


-2.5 


2.9 


2.7 


4.5 


Body mass index^ (kg/m^) 


-0.04 


0.62 


1.05 


0.77 


Overweight or obese (BMI > 25)® (%) 


-1.0 


4.5 


12.4 ** 


5.7 


Current smoker (%) 


-1.4 


3.2 


0.8 


4.3 



(continued) 
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Appendix Table E.3 (continued) 



SOURCES: MDRC calculations from the Opening Doors 12-Month Survey and Post-Katrina Modules. 

NOTES: Rounding may cause slight discrepancies in sums and differences. 

A two-tailed t-test was applied to differences between research groups. Statistical significance levels are 
indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. 

Estimates are adjusted by research cohort and campus. 

“An 8 -item scale about the presence of social support; response categories range from 1 = "strongly disagree" 
to 4 = "strongly agree." Items are averaged. 

'’Performed any unpaid volunteer or community service work since random assignment. 

“A 4-item scale about feelings of social stress; response categories range from 1 = "never" to 5 = "very often." 
Items are averaged. 

‘'A 6 -item scale about nonspecific psychological distress; response categories range from 0 = "none of the 
time" to 4 = "all of the time." Items are summed. 

“An indicator if the K 6 Screening Scale measure of psychological distress exceeds 12. 

*BMI = weight in kilograms divided by height in meters squared. 

si = BMI of 25 or greater; 0 = BMI of less than 25. Standard weight-status categories associated with BMI 
ranges for adults: underweight less thanl8.5; normal weight = 18.5 to 24.9; overweight = 25.0 to 29.9; and obese = 
30 or greater. 
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About MDRC 



MDRC is a nonprofit, nonpartisan social policy research organization dedicated to learning 
what works to improve the well-being of low-income people. Through its research and the ac- 
tive communication of its findings, MDRC seeks to enhance the effectiveness of social and 
education policies and programs. 

Founded in 1974 and located in New York City and Oakland, California, MDRC is best known 
for mounting rigorous, large-scale, real-world tests of new and existing policies and programs. 
Its projects are a mix of demonstrations (field tests of promising new program approaches) and 
evaluations of ongoing government and community initiatives. MDRC’s staff bring an unusual 
combination of research and organizational experience to their work, providing expertise on the 
latest in qualitative and quantitative methods and on program design, development, implementa- 
tion, and management. MDRC seeks to learn not just whether a program is effective but also 
how and why the program’s effects occur. In addition, it tries to place each project’s findings in 
the broader context of related research — in order to build knowledge about what works across 
the social and education policy fields. MDRC’s findings, lessons, and best practices are proac- 
tively shared with a broad audience in the policy and practitioner community as well as with the 
general public and the media. 

Over the years, MDRC has brought its unique approach to an ever-growing range of policy 
areas and target populations. Once known primarily for evaluations of state welfare-to-work 
programs, today MDRC is also studying public school reforms, employment programs for ex- 
offenders and people with disabilities, and programs to help low-income students succeed in 
college. MDRC’s projects are organized into five areas: 

• Promoting Family Well-Being and Child Development 

• Improving Public Education 

• Promoting Successfirl Transitions to Adulthood 

• Supporting Low-Wage Workers and Communities 

• Overcoming Barriers to Employment 

Working in almost every state, all of the nation’s largest cities, and Canada and the United 
Kingdom, MDRC conducts its projects in partnership with national, state, and local govern- 
ments, public school systems, community organizations, and numerous private philanthropies. 




